Skip to content

fork: follow each stable upstream release with the patch series - #4

Merged
scuttletoby merged 13 commits into
mainfrom
fork/follow-upstream
Nov 12, 2026
Merged

fork: follow each stable upstream release with the patch series#4
scuttletoby merged 13 commits into
mainfrom
fork/follow-upstream

Conversation

@scuttletoby

@scuttletoby scuttletoby commented Nov 12, 2026 •
edited
Loading

Copy link
Copy Markdown
Owner

Keeps the fork's patch series on upstream's newest stable release and publishes it with no one stepping in. When something needs a person, the run fails and its summary gives the reason.

fork-follow.yml runs daily at 06:23 UTC, and can also be started by hand. A Guard job runs first (see below). Then .github/scripts/fork-follow.sh plans one of five actions:

Action When What happens
none The newest v<tag>-fork.N already points at the series head Nothing is built. A leftover fork-follow/candidate branch is deleted. The report closes an open follow issue.
release Same upstream tag, but the series head has no fork release yet (for example, a fork PR merged) fork-release.yml (#2) builds and publishes the head as it is.
rebase Upstream has a newer stable tag (vX.Y.Z, no nightly or preview) The series is rebased onto it. Patches upstream now contains become empty and are dropped. Then the schema check runs, the result is pushed as fork-follow/candidate, it is released, and the series branch moves to it.
advance An earlier run published a release of exactly these files but could not move the branch Nothing is built. Only the branch moves.
stage The release workflows (fork-release.yml and every workflow it calls) of the run's commit differ from the candidate's. That happens when upstream changed them, or on a re-run or queued run of an older commit. It is checked on both the release and the rebase path. The candidate is pushed and the run stops. A run uses the reusable workflows of its own commit, so building now would mix two versions of the release steps. The summary prints the gh workflow run fork-release.yml --ref fork-follow/candidate … command. After that release, the next follow run finds it (and advances to it).

The plan refuses, with a reason, in these cases:

  • a conflict. It names the stopped patch and its files, aborts the rebase and leaves the series as it is. The advice says to start the follow workflow once after the manual push.
  • merge commits in the series (a rebase would flatten them and lose a change that lives only in a merge)
  • upstream commits inside the series
  • a schema change. A crash of the schema check is reported as a crash, not as a schema change.
  • a failed fetch
  • the series' upstream tag was deleted or moved upstream
  • the run's own commit is missing from the fork
  • any other failing line (its line number is reported)

Guard: the token's isolation is checked, not assumed. .github/scripts/fork-follow-guard.sh runs in the Guard job, which has no Environment. It runs before any job that gets FORK_PUSH_TOKEN, and it refuses in these cases:

  • A repository or organisation secret named FORK_PUSH_TOKEN exists. A workflow file on any pushed branch can read such a secret, and a job in an Environment falls back to it when the Environment has none. Outside an Environment, secrets.FORK_PUSH_TOKEN can only be such a secret. So the job passes only secrets.FORK_PUSH_TOKEN != '' (true/false), never the value. GITHUB_TOKEN has no permission that lists secrets, so this is the check GitHub allows.
  • The Environment fork-push does not exist, or it allows any branch, or it allows "Protected branches only". The API cannot prove that this means only main.
  • Its deployment rules are anything other than exactly one rule: branch main, with no pattern and no tag rule. The guard reads them with GITHUB_TOKEN (actions: read).

Each refusal says what to change under Settings. A refusing guard also means the plan job never runs, so GitHub does not auto-create an empty fork-push Environment.

The guard counts for one run attempt only. "Re-run failed jobs" and "Re-run job" keep the Guard job's earlier result without running it again, and the settings may have changed since. The guard outputs the attempt it passed in, set by a step after the guard script. Plan and the branch move run only when that is the current github.run_attempt. The check is a job-level if: (with the implied success()), so a skipped job never reaches the Environment. The summary then says to use "Re-run all jobs" or start a new run. Right before the branch move, which can come an hour or more after the guard, the move job checks the Environment's rule again (fork-follow-guard.sh --environment-only). The repository secret cannot be checked from inside the Environment.

Pushes. Every push goes through .github/scripts/fork-follow-push.sh with FORK_PUSH_TOKEN. Nothing falls back to another token.

  • The plan checks the token first. It fails if the token is missing, refused by GitHub (expired or revoked), or unable to push.
  • To move the series branch, the script first keeps the old head as the tag fork-follow/pre-<tag>-<sha12>. Then it pushes with a lease against the planned head. If the branch is already at the target (a re-run), that counts as success. If the branch moved elsewhere (a fork PR merged mid-run), the push is refused, and the next run rebases again with that change.
  • The token reaches git only as an HTTP header in GIT_CONFIG_*. It is never in a URL, .git/config or an error message. Every checkout uses persist-credentials: false.
  • The release job gets the signing secrets by name, never secrets: inherit. Every action is pinned by commit SHA.

Upstream workflows stay off. .github/scripts/fork-workflows.sh runs after the candidate push, right before the branch move, after it, and again in the report. Each time it:

  • disables every active workflow under .github/workflows that the series did not add
  • cancels every unfinished run of such a workflow, whatever started it (push, pull_request, pull_request_target, schedule …) and for any commit
  • names the upstream runs that finished since the run began, with their event and conclusion (they ran before a sweep could stop them)
  • names the upstream workflow files in the tree that GitHub does not list yet. GitHub registers a workflow only once an event reaches its file, so these cannot be disabled until then. A sweep disables one once an event reaches it.
    • Files new on the target release (the plan's new_workflows: added between the old and the new upstream base; a rename counts as new) get a warning and their own row.
    • Unlisted files from earlier releases go in a collapsed note, with no warning. Most of them run only on events the fork never sends (workflow_call, workflow_dispatch, workflow_run), so they stay unlisted. In this fork today that is about 12 files.

The summary shows finished runs and new unlisted files as their own rows.

The branch move waits for new upstream workflows. The Environment ties the token to the branch main, not to fork-follow.yml, so any workflow file on main may ask for fork-push. Moving main also fires its push workflows before a later sweep can stop them. So the move job sweeps first, and refuses to move main while an upstream workflow file that is new on this release is in the candidate and GitHub does not list it. The reason names the files and leaves it to a person: read them; if they are harmless, move main by hand (the command is given) and start the follow workflow right after; otherwise add a fork patch that deletes them. Until then each run plans advance and refuses again.

Report. The Report job runs on every run. It reads the series branch, the candidate branch, the backup tags and the releases back from the repository, rather than inferring them from job results. So it never says "nothing was published" after a release was published. The run's summary is the report, and the run fails whenever a person is needed. If the repository has Issues enabled, the job also opens or comments on one issue, "Fork cannot follow upstream", and closes it on the next good run. If Issues are disabled, it says so and opens nothing.

Owner setup before this can run

  1. Environment. Create the Environment fork-push (Settings > Environments > New environment).
    • Under Deployment branches and tags, pick "Selected branches and tags" and add one rule: branch main. Add nothing else.
    • Add the secret FORK_PUSH_TOKEN there: a fine-grained token for this repository only, with Contents and Workflows read and write.
    • GITHUB_TOKEN cannot replace this token, because it may not push commits that change workflow files.
  2. Delete the repository secret FORK_PUSH_TOKEN (Settings > Secrets and variables > Actions > Repository secrets). It exists today, and while it does, the guard refuses every run.
  3. Allow only squash and rebase merges. The plan refuses merge commits in the series.
  4. Decide on Issues. Today they are disabled, and then the run's status and summary are the only report.
  5. If main gets branch protection or a ruleset, allow the token's user to bypass it. Otherwise the branch move is refused, and the next run takes advance again.
  6. Fork main must be a patch series: v0.1.9 plus the fork's own commits, with no merge commits. Today it equals upstream main, and the plan refuses that state and says so. Right after force-pushing the rebuilt main, start the workflow once (gh workflow run fork-follow.yml --repo scuttletoby/t3code). That push fires upstream's push: main workflows, and the run disables them, cancels their runs and names the ones that already finished.

Today only the fork's own workflows are active (fork-checks, fork-release, fork-tests).

Merge order. Merge #2 (fork-release.yml) and #6 (fork-tests.yml, which #2 calls) first. Merged before them, this workflow file is invalid, and every run fails at load with no report. This branch sits on #3 (the schema check).

Follow-up, not in this PR. stage needs a person each time upstream changes release-desktop.yml, which happened in 4 of the last 7 upstream releases. The follow job could start fork-release.yml --ref fork-follow/candidate itself (workflow_dispatch with GITHUB_TOKEN and actions: write), and the next run would advance as it does now. This is left for a separate change (DI-1 round 2, minor 4).

Tested. .github/scripts/fork-follow.test.sh has 87 cases. The plan job runs it before anything else. The cases cover:

  • each action and refusal above, plus empty or tagless history, against local git remotes and a fake gh
    • including stage on the release path for a re-run of an older commit
    • a schema-check crash told apart from a refusal
    • no stray reason from a failing command substitution
  • the guard: repository secret present, answer missing, Environment missing, any branch allowed, protected branches only, an extra tag rule, a pattern rule, a series branch that is itself a pattern (with the matching rule), no rule, an unreadable Environment, and the passing case; --environment-only passing without the secret answer and still refusing an open Environment
  • the plan's new_workflows: a file added and a file renamed on the target release; none on the same base
  • the push script: token missing (with and without the Environment), refused or working; candidate, and a refused candidate push; advance, its re-run and its lease refusal; cleanup
  • the workflow sweep. The fake filters runs by head_sha and status as GitHub does. Covered: runs from other events and commits are cancelled; finished runs are named, but not own or cancelled ones; unlisted files new on the release are named with a warning, older ones apart without one; before the move, a new unlisted file refuses it (after the listed ones are disabled), while a listed new file or older unlisted ones do not.
  • the report, built from repository state: Issues on and off, the guard's refusal, finished runs and unlisted files (older ones collapsed), and a guard from an earlier attempt (plan skipped; or released but the move skipped), while a re-run of the report alone after a good run passes
  • static checks on the workflow file:
    • secrets passed by name
    • the token only in push steps and only in fork-push jobs
    • the guard job has no Environment and sees only != ''
    • the plan waits for the guard, and its only job-level if: is this attempt's guard (always(), !cancelled() or failure() there turns it red)
    • the move waits for the plan's success and this attempt's guard; the attempt output is set only after the guard script
    • before the move: the Environment check again, then the sweep with REQUIRE_NEW_OFF, and the move step has no if:
    • no persisted credentials, pinned actions
    • a report on every run, sweeps after each push, cleanup on none

actionlint 1.7.7 (with shellcheck) passes on this workflow next to main's fork-release.yml.

Not tested

  • No GitHub run yet. Scheduled and dispatched runs need the workflow on the default branch, and fork-release.yml is not on this branch.
  • Re-run behaviour is from GitHub's documentation, not seen in a run: that "Re-run failed jobs" keeps the outputs of jobs it does not re-run and raises github.run_attempt.
  • Whether the candidate push registers a new upstream workflow whose on: push names only main (most likely not, so such a release needs a person each time).
  • A workflow file upstream changed (not added) that stays unlisted does not hold up the move. One that gains a push trigger and asks for fork-push would still run once on main.
  • Not checked against GitHub:
    • whether an expired token on a public repository makes ls-remote fail
    • whether push --dry-run checks push access
    • whether secrets.FORK_PUSH_TOKEN != '' outside an Environment also sees an organisation secret (GitHub documents it; this repository is not in an organisation)
    • whether an Environment's "Allow administrators to bypass" lets an admin's run from another branch reach the secret. The guard does not check can_admins_bypass.
  • Checked read-only against GitHub: the guard's answers for a missing Environment (this fork) and for one with an extra tag rule (upstream's Production), plus the sweep's contents and runs queries.

🤖 Generated with Claude Code

@github-actions github-actions Bot added vouch:trusted PR author is trusted by repo permissions or the VOUCHED list. size:L labels Nov 12, 2026
@scuttletoby

scuttletoby commented Nov 12, 2026

Copy link
Copy Markdown
Owner Author

lens: ops+security DI-1 · reviewer: opus high (t3lens-di1) · read at: cd488d4
covered: .github/scripts/fork-follow.sh (all), .github/workflows/fork-follow.yml (all); read for context: fork-release.yml from #2 (48526ae, all), fork-schema-check.cjs (#3, only how it is run), release-desktop.yml secrets and permissions, the triggers of every upstream workflow in the tree, repo merge settings and branch protection.
checks: Ran fork-follow.sh against a local fake upstream (tags v0.1.0, v0.2.0) and fork. (1) Series with a merge commit that carries an extra change: rebase "succeeds", the change is gone, the merge is listed in dropped=. (2) Rerun after a published release whose advance did not happen (v0.2.0-fork.1 tagged at the candidate, main not moved): action=rebase again with a new candidate SHA, so a new release follows. Traced by reading: conflict path (rebase --abort, nothing pushed, report names the stopped patch and its files, one issue, a comment per day while it lasts); lease at fork-follow.yml:119 (server-side compare against the plan's checkout of main, so a fork PR merged mid-run, a human force-push or "re-run failed jobs" days later all refuse); candidate failing tests or build (main untouched, candidate branch overwritten next run); an existing tag (fork-release preflight picks N+1); an older upstream tag published later (target is the highest version, sort -rV, so no downgrade); no stable tag (fails with a report); fork main = upstream main today (470 commits past v0.1.9: series_count != own_count, so it refuses with a report; the guard is right, and the workflow file is itself a fork commit, so main can never equal upstream while this runs).
State question: a rerun is safe for main (lease) but not for releases (finding 2). Stopping half-way: before publish leaves nothing; after publish but before advance leaves a published release, main not moved, and a wrong report (finding 2). Legacy history with merge commits loses merge-only changes (finding 1). Empty or upstream-only history is refused with a report.
FORK_PUSH_TOKEN: needs a fine-grained token for this repository only, with Contents read/write and Workflows read/write (Metadata read comes with it). If a ruleset ever protects main, the token's user must be allowed to bypass it. Where it can reach: only the plan and advance jobs (checkout persists it in .git/config); the triggers are schedule and workflow_dispatch only, so fork PRs cannot reach it. secrets: inherit (line 93) also hands it to fork-release.yml and upstream's release-desktop.yml, but neither names it, so no step gets it. In the plan job no upstream code runs after the checkout (the schema check is the fork's own file and only runs git). Untrusted input in run: lines: base_tag is limited to vX.Y.Z (fork-follow.sh:44), previous_head and candidate are SHAs, and commit subjects go through env. No injection found.
Safe first run: (1) push a backup ref of the current main; (2) build main as v0.1.9 plus only the fork commits (git rebase --onto v0.1.9 dc032d7 main), resolve conflicts, and check the patches don't need upstream commits after v0.1.9 (the release-desktop.yml inputs fork-release uses, i.e. cli_archive, launch_check, verify_runner, resource_key and relay_client_tracing, do exist at v0.1.9); (3) set FORK_PUSH_TOKEN and turn off merge commits (finding 1); (4) force-push main and rebase the open PRs (#5) onto it; (5) run fork-release.yml by hand with publish=false; (6) dispatch fork-follow, expecting action=release and v0.1.9-fork.1.
finding: major .github/scripts/fork-follow.sh:107-108 (and :117, fork-follow.yml:171) A series with merge commits -> the rebase (no --rebase-merges, no merge check) flattens them. A change that lives only in a merge commit (an edited or "evil" merge) is silently gone. The merge is also listed in dropped= and reported as "Dropped because upstream now has them", which is false. Then advance force-pushes main, and the old head stays reachable only if a release tagged it. The repo allows merge commits (allow_merge_commit=true) and main is unprotected. Tested: evil.txt from the merge was missing after the rebase, exit 0, action=rebase. Fix: refuse when git rev-list --merges "$base..HEAD" is non-empty (with a report), turn off merge commits on the fork, and push a backup ref of previous_head (e.g. refs/tags/fork-follow/pre-) before advance. (high)
finding: major .github/workflows/fork-follow.yml:98-121 with :159 Publish succeeds, then advance fails (lease refused because a fork PR merged mid-run, or a ruleset blocking force-push, which the PR text expects) -> the issue says "A test, build or publish job failed for vX; see the run. Nothing was published.", but vX-fork.N is published and marked latest, and auto-updates pick it up. On the next run released_head (fork-follow.sh:90) never matches the new rebase SHA, so it rebases and publishes again. If advance keeps failing, that is a new 4-platform release (and update) every day, each reported as "nothing published". Tested: the rerun gives action=rebase with a new candidate. Fix: the report names the published tag and the advance failure. Before releasing, plan checks whether a fork release of the target exists whose commit's tree equals the candidate's (or whose range-diff is empty), and if so advances to it instead of building again. (high for the wrong report, medium for the daily loop)
finding: minor .github/scripts/fork-follow.sh:59,70,105 A failed git fetch or ls-remote (network, a tag deleted between list and fetch) exits through set -e without writing $REPORT -> the issue gets the generic "test, build or publish job failed" text, which names the wrong cause. Fix: wrap the fetches with fail "...". (high)
finding: minor .github/scripts/fork-follow.sh:66-76 Upstream deletes (yanks) the tag the fork already follows -> base_tag falls back to the older tag, and the report says "Reset it to plus the fork's own commits". That advice would put main below the installed fork release. Fix: say that the tag the series sits on is gone. (medium)
finding: minor .github/workflows/fork-follow.yml:92 The local reusable workflow (and the release-desktop.yml it calls) comes from the run's commit, which is main before the rebase, not from the candidate -> the first build after following a new upstream tag uses the previous upstream's release-desktop.yml with the new upstream's build scripts. When inputs or steps changed between tags, that build fails (reported, safe) or differs from upstream's own build. Worth a line in the docs, or a check that release-desktop.yml is the same in both. (medium)
nits (optional): actions use tags, not SHAs (lines 41, 78, 105, 135), although upstream pins some by SHA (desktop-macos-preview-publish.yml:58) and these jobs hold the push token. Release passes secrets: inherit where it could name only what it needs. The PR does not document how a human resolves a conflict (rebase main onto the tag by hand and force-push). Owner follow-up (a) (disable upstream workflows that arrive through a rebase) is not in this PR; note that advance pushes with a PAT, so push:main workflows do fire (ci.yml has no repository guard; disabled for now, but a new one would arrive enabled).

@scuttletoby

scuttletoby commented Nov 12, 2026

Copy link
Copy Markdown
Owner Author

lens: ops+security DI-2 · reviewer: opus high (t3lens-4-di2) · read at: cd488d4
covered: #4's own commit cd488d4 in full: .github/workflows/fork-follow.yml, .github/scripts/fork-follow.sh. Read for the calls: fork-release.yml from #2 (48526ae, in full), the secrets and triggers in release-desktop.yml and the other workflows on main, and the fork's repo settings (has_issues, merge methods, branch rules, workflow states). Design doc read first, including both rounds of owner answers.
checks:

  • Ran fork-follow.sh against a fake upstream and fork. The series held two fork PRs merged with merge commits, and one merge commit carried its own change. Result: action=rebase; the file added in the merge commit is missing from the candidate; both merges are reported as "dropped". See major 3.
  • Ran gh issue list --repo scuttletoby/t3code .... It prints "the 'scuttletoby/t3code' repository has disabled issues" and exits 1. has_issues=false. See major 1.
  • Traced the hardest case: the release publishes, then advance fails (a lease mismatch, or main protection the token can't bypass). See major 2.
  • State question, per case:
    • Rerun of failed jobs after advance failed: advance is retried with the same lease. That is fine.
    • Full rerun, or the next daily run: plan rebases again and gets a new SHA (new committer date). fork-release counts the existing tags, so it publishes v-fork.N+1. A fork.N release can point at a commit that main never gets. Nothing is lost, because the tag keeps it.
    • A stop half-way through the rebase: rebase --abort runs and the report names the patch and files. That is fine.
    • The release fails after its draft: no advance runs. fork: release workflow for v<upstream>-fork.N on GitHub-hosted runners #2 deletes the draft on the next run. "Nothing was published" is true here.
    • Two runs at once: concurrency: fork-follow, no cancel. A queued run reads main afresh and the lease guards the push. That is fine.
    • An older upstream tag published later: sort -rV ignores it. That is fine.
    • A moved or deleted upstream tag: the base falls back to the previous tag. If upstream main still has the old commits, it refuses (see the minor below). If upstream also rewrote main, those commits count as "own" and get replayed.
    • Empty fork history: the script fails with "contains no stable tag". That is safe.
  • Untrusted input in run: lines: base_tag comes only through the sed whitelist v[0-9]+.[0-9]+.[0-9]+$, and candidate and previous_head are SHAs. DROPPED (commit subjects) reaches the shell only through env. Release bodies are never read. Triggers are schedule and workflow_dispatch only, with no pull_request(_target), so fork PRs can't reach the token. Release tags are made with GITHUB_TOKEN, so release.yml's v*.*.* push trigger does not fire.
  • The "refuse while main equals upstream main" guard (fork-follow.sh:47-53):
    • On today's main dc032d7 (upstream main, past v0.1.9) it refuses: upstream commits after v0.1.9 are not "own". That is correct.
    • The one gap: commits upstream later removes from its default branch become "own".
    • The refusal's advice "Reset it to $base_tag plus the fork's own commits" is right for the first run.
      finding: major .github/workflows/fork-follow.yml:150 The fork has Issues disabled (has_issues=false; gh issue list exits 1, shown above), so the Report job fails at its first command on every failure: a conflict, a schema change, a red test, or the "main equals upstream" refusal the first run will hit. No issue is ever opened or commented. The only signal left is a red scheduled run. Fix: enable Issues in the setup list and check for them in plan, or fail with a ::error:: and a step summary when issues are off. (high)
      finding: major .github/workflows/fork-follow.yml:101-121,143,159 The release publishes, then advance fails. Causes: the lease refuses (a fork PR merged during the 30-60 min build), or main gets the branch protection the setup text mentions and the token can't bypass it. The report then says "A test, build or publish job failed for vX; see the run. Nothing was published." That is false: vX-fork.N is published and marked latest, and T3 Update offers it. If the cause persists (protection), each daily run rebases main again (still on the old base), publishes vX-fork.N+1 from a commit main never gets, and claims nothing was published. Fix: make the report text depend on needs.release.result. In plan, when the latest fork tag of target_tag is a released candidate that main lacks, plan only the advance (or refuse) rather than build again. (high)
      finding: major .github/scripts/fork-follow.sh:84-85,94 Merge commits in the series are flattened without a report. The fork allows merge commits (allow_merge_commit=true), and a fork PR merged with "Create a merge commit" after a conflict resolution in the GitHub UI keeps that resolution only in the merge commit. Without --rebase-merges, git rebase drops merge commits, so the change is lost from the candidate and from the moved main. The merges then show up in dropped, and the success note says "Dropped because upstream now has them", which is wrong. Reproduced: the file added in the merge commit is missing after rebase. Fix: refuse when git rev-list --merges "$base..HEAD" is non-empty, saying so in the report, and allow only squash or rebase merges in the fork. (high)
      finding: major .github/workflows/fork-follow.yml:115-121 Owner follow-up (a) from 2026-11-12 says the follow-upstream job disables every workflow that is not the fork's own (an allowlist) and says so. That is missing. advance pushes main with FORK_PUSH_TOKEN, and unlike a GITHUB_TOKEN push this triggers push: main workflows. Any new upstream workflow file that arrives through the rebase is enabled and runs on that push, without notice. (medium: high that the gap is real; medium on whether it was meant for this PR or a later one)
      finding: minor .github/workflows/fork-follow.yml:93 secrets: inherit hands FORK_PUSH_TOKEN (contents and workflows write) to fork-release.yml, and from there to release-desktop.yml. That file is upstream-authored and reaches main through each rebase. Neither file needs the token. Pass the signing secrets by name instead. Also, the plan checkout (line 46) persists the PAT in .git/config for the whole plan step when only the push needs it; check out with github.token and use the PAT only in the push. (high)
      finding: minor .github/workflows/fork-follow.yml:159 Plan failures outside fail() report as "A test, build or publish job failed ... Nothing was published". Examples: a network or fetch error under set -e, or a rejected candidate push at line 74 (for example a token without Workflows write). The text points at the wrong stage and has no reason. Write a REPORT for those too, for example a trap ERR in fork-follow.sh, plus one for the push. (high)
      finding: minor .github/scripts/fork-follow.sh:51-52 When upstream moves or deletes the tag the series sits on, the base falls back to the older tag. The refusal then advises "Reset it to plus the fork's own commits", which would drop the newer base. Name the moved tag instead (for example: a tag the series once contained is no longer an ancestor). (medium)
      finding: minor .github/workflows/fork-follow.yml:126 A failure issue stays open forever if the problem is then fixed by hand, for example a manual fork-release run, because the next follow run is action=none and Report is skipped. (medium)
      finding: minor .github/workflows/fork-follow.yml:121 git push origin --delete fork-follow/candidate || true fails without a report. It's harmless, because the next run force-pushes over the branch, but add an ::warning::. (high)
      Token minimum: a fine-grained token on scuttletoby/t3code only, with Contents read/write, Workflows read/write and Metadata read. Nothing else, and no other repos. Alternative: a write deploy key for this one repo. It has no API access and does not expire, and SSH pushes can change workflow files. Branch protection on main needs a bypass for that identity, or major 2 loops.
      Safe first run:
  1. Enable Issues.
  2. Allow only squash and rebase merges.
  3. Build main as v0.1.9 plus the PR commits fork: the server and CLI update from the desktop app's repository #1-Auto-update: server updates itself when quiet, desktop installs on quit #5, rebased linearly and with no merge commits, and force-push it once by hand.
  4. Add FORK_PUSH_TOKEN.
  5. Dispatch once by hand before upstream ships v0.1.10. That run must plan action=release (no rebase, no advance) and publish v0.1.9-fork.1.
  6. Only then rely on the schedule and add any protection with a bypass.
    Style nits (optional): previous_tag is written to GITHUB_OUTPUT but never mapped as a job output.

@scuttletoby scuttletoby mentioned this pull request Nov 12, 2026
Merged
@scuttletoby scuttletoby mentioned this pull request Nov 12, 2026
Merged
@scuttletoby

scuttletoby commented Nov 12, 2026

Copy link
Copy Markdown
Owner Author

fix round 1: rebased onto #3's new head (head d92e84253d)

No change to this patch (range-diff identical). The follow-mode schema check over origin/main..HEAD exits 0. Note for #3's ruleset: this workflow force-pushes main with FORK_PUSH_TOKEN, so that token's owner must be a bypass actor.

🤖 Generated with Claude Code

@scuttletoby

scuttletoby commented Nov 12, 2026

Copy link
Copy Markdown
Owner Author

Fix round 1 for the lens reviews DI-1 (6819901664) and DI-2 (6820054224)

  • Head: 62216fcc84, rebased onto fork: CI check that refuses database schema and migration changes #3's fix round 2 (a7328e42ae). The PR's own first commit is now 8c32db60f4, which is d92e84253d rebased.
  • Fix range: 8c32db60f4..62216fcc84 (4f18a80785, c21e315b0f, 62216fcc84).
  • Tests: .github/scripts/fork-follow.test.sh, 44 cases, all green on the new head. The plan job runs them first.
  • "Red on old" means the same suite run with FORK_FOLLOW_DIR/FORK_FOLLOW_WORKFLOW pointed at the pre-fix scripts and workflow (d92e84253d). The old Report step's shell was extracted into a script for that run. A test that is green on both is marked guard: the old code already behaved correctly, and the test keeps it that way.

Findings

# Finding Fix Test (old → new)
DI-1 major 1, DI-2 major 3 Merge commits in the series were flattened and listed as "dropped" The plan refuses rev-list --merges with a report naming the merges, and leaves the series as it is. Before moving the branch, the old head is kept as tag fork-follow/pre-<tag>-<sha12>. Allowing only squash and rebase merges is in the owner setup. plan: merge commits in the series are refused and named red → green. push: advance moves the series and keeps the old one as a tag red → green.
DI-1 major 2, DI-2 major 2 Published but not moved: the report said "Nothing was published", and every run rebuilt and republished (a) The Report reads branches, tags and releases back from the repo. (b) New action advance: if a published fork release of the target has the candidate's tree, only the branch moves and nothing is built. report: published but not moved -> says both, never 'nothing was published' red → green. report: a failed release with no tag -> 'none published' red → green. plan: a published release of the rebased files exists -> advance to it, build nothing red → green. plan: a release of different files does not count -> rebase guard.
DI-2 major 1 Issues are disabled, so the Report died at gh issue list The run's summary is the report, and the run fails when a person is needed. An issue is opened, commented on or closed only when has_issues is true; otherwise the summary says so. report: Issues disabled -> the reason is in the summary, no issue call, the run fails red → green. report: Issues enabled -> one issue is opened on failure red → green.
DI-2 major 4, DI-1 nit Upstream workflows that arrive through a rebase stay enabled fork-workflows.sh disables every active .github/workflows file that the series did not add. It also cancels their unfinished runs for the pushed SHA. It runs after the candidate push, after the branch move, and in the Report. workflows: an active upstream workflow is disabled and named … red → green. workflows: unfinished runs … are cancelled … red → green. workflows: an empty own list disables nothing (refusing to guess) red → green. workflows: a failed listing fails with a reason red → green. workflow: upstream workflows are swept after each push red → green. report: names each disabled upstream workflow once red → green.
DI-1 minor 3, DI-2 minor 6 A failed fetch, any other set -e exit, or a refused candidate push gave no reason The fetches use fail with git's output. A trap ERR reports the failing line. Every push in fork-follow-push.sh writes git's reason. plan: a failed upstream fetch writes a reason naming it red → green. plan: an unexpected failure writes a reason naming the line red → green. push: a refused candidate push fails with git's reason red → green.
DI-1 minor 4, DI-2 minor 7 A deleted or moved upstream tag gave the advice "reset to " The report names the deleted or moved tag that the last fork release was built on. plan: upstream deleted the tag … -> named, not 'reset to v0.1.9' red → green. plan: upstream moved the tag … -> named red → green.
DI-1 minor 5 The release ran the old upstream's reusable workflows on the new tag New action stage: if fork-release.yml or any workflow it calls differs between the run's commit and the candidate, the candidate is pushed and nothing is built. The summary prints the gh workflow run fork-release.yml --ref fork-follow/candidate … command, and the next run advances to that release. plan: changed release workflows -> stage, with the command … red → green. report: stage fails the run with the reason red → green.
DI-2 minor 5, DI-1 nit secrets: inherit, and the PAT persisted in .git/config The release gets the 13 signing secrets by name. Every checkout uses persist-credentials: false. The token is used only in fork-follow-push.sh steps, as an HTTP header in GIT_CONFIG_*, never in a URL or the config. workflow: release gets exactly the signing secrets by name, no 'secrets: inherit' red → green. workflow: FORK_PUSH_TOKEN reaches only fork-follow-push.sh steps, never a checkout red → green. workflow: every checkout drops its credentials red → green.
DI-2 minor 8 A follow issue stayed open after a fix by hand The Report runs if: always(). On action none it closes an open follow issue. report: action none closes an open follow issue and the run passes red → green. workflow: the report runs on every run red → green.
DI-2 minor 9 --delete … || true without a report cleanup prints ::warning:: and never fails the run. push: cleanup deletes the candidate branch guard (covers the delete only; the warning path has no test).
DI-1 nit Actions pinned by tag Every action is pinned by commit SHA. workflow: every action is pinned by commit red → green.
DI-1 nit No documented way to resolve a conflict by hand The workflow header and the conflict report give the commands. plan: a conflict names the stopped patch and its file and aborts the rebase guard.
DI-2 nit previous_tag was not a job output It is now mapped as a plan output and used in the report. none
Token minimum (both) – FORK_PUSH_TOKEN is checked on every run before anything else: missing, refused by GitHub, or unable to push (push --dry-run). push: a missing FORK_PUSH_TOKEN fails and says so red → green. push: a token GitHub refuses (expired) fails and says so red → green. push: a working token passes the check red → green.
Orchestrator, after review A repository secret is readable by a workflow file on any pushed branch The plan and advance jobs, the only ones that read the token, run in environment: fork-push. If the token is empty there, the check fails, and the summary names the Environment and says to limit it to main. Creating it is owner step 1 in the PR body. push: a missing FORK_PUSH_TOKEN names the Environment that should hold it red → green. workflow: the jobs that read FORK_PUSH_TOKEN, and only they, run in the Environment fork-push red → green.

I did not take the deploy-key alternative (DI-2): the plan and Report also need API calls, and a deploy key cannot make them.

State question: one test per row

State Before Now Test (old → new)
Re-run after the branch moved The lease against the planned head refused, so the run failed Already at the target counts as success (moved=already), and nothing is pushed push: advance again after it moved (re-run) succeeds without a push red → green
Re-run or next run after the branch moved elsewhere (a fork PR merged mid-run) Lease refused, with a generic report Lease refused with a reason, the branch left as it is, and the next run rebases with that change push: advance refuses when the branch moved since the plan, and leaves it red → green
Half-applied: release published, branch not moved Rebuilt, republished as fork.N+1 and reported "nothing published" Plan picks advance to the published release, and the report says it was published and not moved plan: a published release of the rebased files exists -> advance … red → green; report: published but not moved … red → green
Legacy history with merge commits Flattened, merge-only changes lost, merges listed as dropped Refused and named, series left as it is plan: merge commits in the series are refused and named, the series left as it is red → green
Empty or tagless history Refused (safe) Same, still with a reason and no action plan: upstream has no stable tag -> refused … guard; plan: the series contains no stable upstream tag -> refused … guard
Upstream tag moved or deleted Wrong "reset to " advice Names the moved or deleted tag plan: upstream deleted the tag … red → green; plan: upstream moved the tag … red → green

Unproven

  • No GitHub run yet. Scheduled and dispatched runs need the workflow on the default branch, and fork-release.yml (fork: release workflow for v<upstream>-fork.N on GitHub-hosted runners #2) is not on this branch. actionlint is clean except for that missing file.
  • Not checked against GitHub:
    • whether ls-remote with an expired token on a public repository fails (it could read anonymously)
    • whether push --dry-run really checks push access
    • whether workflow-run path carries an @ref suffix (it is stripped if present)
  • Environment fork-push:
    • Its behaviour before the owner creates it is not checked. GitHub makes an empty Environment with no branch rule at the first run that names it.
    • Until the token moves there, a repository secret of the same name still reaches these jobs.
  • A new upstream workflow can be disabled only after the push that registers it. A run it starts is cancelled at the next sweep, so it may start briefly.
  • stage needs a person to run the printed release command.
  • Live fork today: relay-preview.yml and web-preview.yml are active upstream workflows, and the first follow run disables them. has_issues=false and allow_merge_commit=true. I changed no repository settings.

@scuttletoby

scuttletoby commented Nov 12, 2026

Copy link
Copy Markdown
Owner Author

lens: ops+security DI-1 · reviewer: opus high (t3lens-4-di1-r2) · read at: 62216fc
covered: round 2, the fix range 8c32db6..62216fc (4f18a80, c21e315, 62216fc) in full: .github/workflows/fork-follow.yml, .github/scripts/fork-follow.sh, fork-follow-push.sh, fork-workflows.sh, fork-follow-report.sh, fork-follow.test.sh. Read for the calls: fork-release.yml from #2 (a1bc1cd: workflow_call inputs, the secrets it declares, the published output, target_commitish, the publish job's workflow_sha checkout), and the triggers and repository guards of upstream's workflows at v0.1.9. Live fork state checked read-only: the workflow list and states, has_issues, merge settings, Actions default permissions. Head re-checked: still 62216fc.
checks:

  • Rebase: git range-diff d92e84253d~1..d92e84253d 8c32db60f~1..8c32db60f prints 1: d92e84253 = 1: 8c32db60f. The tree diff d92e842..8c32db6 touches only fork: CI check that refuses database schema and migration changes #3's files (fork-schema-check.cjs and its test, fork-checks.yml). fork-follow.* is the same as in cd488d4.
  • Red→green: bash .github/scripts/fork-follow.test.sh on 62216fc: 44/44 ok. The same suite with FORK_FOLLOW_DIR/FORK_FOLLOW_WORKFLOW pointed at 8c32db6's fork-follow.sh and fork-follow.yml: 36 failed. The 8 that stay green are guards (release, clean rebase, conflict, upstream commits inside, the two tagless cases, "release of different files", cleanup).
  • Every round-1 finding, checked at its root:
    • DI-1 major 1 / DI-2 major 3 (merge commits): fixed. fork-follow.sh:118-125 refuses rev-list --merges before any rebase. fork-follow-push.sh:100-103 pushes the backup tag before main moves. Test red→green.
    • DI-1 major 2 / DI-2 major 2 (published, then not moved): fixed. The report reads refs and releases back (fork-follow-report.sh:42-93). The plan's tree-equality advance (fork-follow.sh:182-187) stops the daily rebuild loop. The lease plus moved=already (fork-follow-push.sh:90-106) makes a re-run safe. Both tests red→green.
    • DI-2 major 1 (Issues disabled): fixed. fork-follow-report.sh:139-143 checks has_issues first; the summary and the run status are the report. Test red→green.
    • DI-2 major 4 / owner follow-up (a) (upstream workflows stay on): implemented, with a gap. See minor 2.
    • Fetch failures and set -e exits without a reason: fixed (fork-follow.sh:44-53, 62-66).
    • Deleted or moved upstream tag: fixed (fork-follow.sh:77-94).
    • Reusable workflows taken from the run's commit: fixed for rebase only, through stage. See minor 1.
    • secrets: inherit and the persisted PAT: fixed. The secrets passed by name match what fork: release workflow for v<upstream>-fork.N on GitHub-hosted runners #2's fork-release.yml declares. Every checkout has persist-credentials: false. The token reaches git only as a GIT_CONFIG_* header, and is masked.
    • Issue never closed: fixed (report has if: always(); action none closes the issue).
    • Silent --delete || true: fixed (::warning::).
    • Pins: every action is pinned by SHA. previous_tag is mapped.
  • Hardest case 1 (stopped after main is force-pushed but before the release): this order can no longer happen in a run. main moves only in advance, which needs release.outputs.published == 'true' (fork-follow.yml:183-187) or an existing release (action advance). A stop after the backup tag but before the move leaves main unchanged. The next run plans advance again, and the backup tag push is a no-op at the same SHA. The only "push main first" path is the manual conflict path; there the next run sees target == base and releases.
  • Hardest case 2 (two runs): concurrency fork-follow without cancelling, so runs are serial, and the plan checks out the series branch fresh. The lease is atomic. A daily run during a person's stage release pushes a new candidate SHA over the branch. The release keeps building the old SHA, and once it is tagged the next run advances to it (same tree). Safe. A queued or "re-run all" run whose github.sha is older is minor 1.
  • Hardest case 3 (conflict): rebase --abort, the report names the patch and its files, nothing is pushed. Test green.
  • Hardest case 4 (merges): refused, test green. allow_merge_commit is still true on the fork (owner setup step 2).
  • Hardest case 5 (new upstream workflows and the race): they cannot be disabled before they run. GitHub returns 404 for a workflow file that no event has reached yet: actions/workflows/{release,stale-needs-more-info,deploy-relay,mobile-dev-client,release-desktop}.yml all 404 today, and only 11 of the ~27 files are listed. A push-triggered file starts a run at the candidate or main push, and the sweep that follows cancels it, so one run can start. The upstream jobs I checked guard on github.repository == 'pingdotgg/t3code' or on the PR's head repo (deploy-relay, release, mobile-dev-client), so that run does little. New upstream workflow files arrived in 5 of the last 7 upstream releases (v0.1.2..v0.1.9), so this happens regularly. Non-push triggers: see minor 2.
  • Hardest case 6 (token): missing, expired or no push rights fail in the first plan step with a reason (token tests green). I could not check two things against GitHub without a token: that ls-remote with a bad header fails on a public repo (GitHub normally returns 401, not an anonymous read), and that push --dry-run asks receive-pack for auth. Environment fork-push before the owner makes it: GitHub creates it empty, with no branch rule. With no secret anywhere, the check fails and names the Environment. With a repository secret of the same name, environment jobs get it (GitHub falls back from Environment to repository secrets), the check passes, and nothing says so. Is that safe? Today in practice, yes: only write-access identities push branches, and upstream files don't name FORK_PUSH_TOKEN. But the isolation the workflow header promises (fork-follow.yml:23-26) is assumed, not enforced. See minor 3.
  • Hardest case 7 (Issues disabled): test green. The run fails, and the summary carries the reason.
  • Hardest case 8 (untrusted input): no ${{ }} in any run: line. All values go through env. base_tag and fork tags pass the regexes at fork-follow.sh:55-56. dropped holds only the series' own subjects (comm -23 against before). Workflow paths and run ids from the API go only into ::error:: text and step outputs (no eval). The backup tag name fork-follow/pre-* does not match release.yml's v*.*.* tag trigger. Release tags are made with GITHUB_TOKEN. No injection found.
  • State question: re-run after a move gives moved=already. Re-run after main moved elsewhere: the lease refuses, with a reason. Published but not moved: the next run plans advance. Draft with no tag: "none published", and fork: release workflow for v<upstream>-fork.N on GitHub-hosted runners #2 removes the draft. Tag with its release deleted: reported "tag exists without a release" (run fails, not silent). Legacy merge history: refused. Empty or tagless history: refused (guards). Moved or deleted upstream tag: named. Upstream release with no fork patches applied yet (fork main == upstream main today): refused, "upstream commits after", test green.
  • Read the author's comment 6822304332 only after the above. Its "Unproven" list matches mine except minor 2's non-push triggers.
    finding: minor .github/scripts/fork-follow.sh:141-158 The reusable-workflow check (the called function and stage, :190-222) runs only on the rebase path. When the run's commit (RUN_SHA = github.sha) is on an older upstream than the series head and target == base, the plan returns action=release, and the release job builds the new upstream with the old upstream's release-desktop.yml. This happens on "Re-run all jobs" of an older run after a later run advanced, or on a run queued behind the one that advanced. Tested: in the suite's fixture, upstream v0.1.11 changes release-desktop.yml, the series sits on v0.1.11, and RUN_SHA is the v0.1.10 series. Result: exit 0, action=release, base_tag=v0.1.11, no stage. Fix: run the same changed-workflows comparison for action release too, and stage (or refuse) when it differs. (high)
    finding: minor .github/scripts/fork-workflows.sh:158-185 The sweep sees only workflows GitHub has registered, and it is silent about the rest. (a) Upstream files that no event has reached yet are not listed (404, shown above), so they are neither disabled nor named. The report then says "Upstream workflows disabled: …" and the run passes, as if every upstream workflow were off. (b) It cancels only runs whose head_sha is the pushed SHA (:171-174), so a run started by pull_request, pull_request_target, workflow_run or schedule is never cancelled. Such workflows run on every matching event until the next daily sweep. Example today: closing or merging a fork PR fires relay-preview-cleanup.yml (pull_request_target closed, unregistered, runs with the base repo's token). relay-preview.yml and web-preview.yml already have 13 runs each from PR events. (c) A run that finished before the sweep (status completed, :173) goes unreported. Whether a push to the default branch registers pull_request-only files I could not verify; if it does not, (b) holds after every follow. Fix: list the .github/workflows files in the pushed tree that are neither own nor registered, and name them in the report (::warning::). Optionally add a small own workflow on pull_request/pull_request_target events that runs the same sweep, so a new trigger is caught at its first run, not up to a day later. (medium)
    finding: minor .github/scripts/fork-follow-push.sh:40-51, .github/workflows/fork-follow.yml:23-26,54,181 Three ways the token can sit outside the isolation the workflow header promises: a repository secret FORK_PUSH_TOKEN, an organisation secret, or the Environment auto-created with no branch rule. In each case the check passes and no report says so. GitHub falls back from Environment to repository secrets. An Environment with no deployment-branch rule hands its secret to gh workflow run fork-follow.yml --ref <any branch>, which runs that branch's YAML with the token. Practical risk today is low (only write-access identities can dispatch or push branches), but the guard is trusted, not enforced. Fix, both checkable with GITHUB_TOKEN: (1) a job with no environment that fails when secrets.FORK_PUSH_TOKEN != '' ("a repository or organisation secret of that name exists; delete it"); (2) in plan, gh api repos/$GITHUB_REPOSITORY/environments/fork-push, refusing unless deployment_branch_policy limits it to SERIES_BRANCH. (high on the mechanism)
    finding: minor .github/scripts/fork-follow.sh:212-221 stage needs a person whenever upstream changes release-desktop.yml. That was 4 of the last 7 upstream releases (v0.1.3→4, 4→5, 6→7, 7→8). So "follow with no one stepping in" holds for under half of them. It reports itself, so nothing is wrong, but the person's step can be automated: the follow job can run gh workflow run fork-release.yml --ref fork-follow/candidate -f ref=<candidate> -f publish=true with GITHUB_TOKEN plus actions: write (workflow_dispatch is exempt from the rule that GITHUB_TOKEN events start no runs), and the next run advances as now. (high on the data)
    nits (optional):
  • fork-follow.sh:127-128 and :175-176 report any non-zero exit of fork-schema-check.cjs as "The series changes the database schema". When I ran it with the script missing, node's MODULE_NOT_FOUND was reported that way. Tell a refusal apart from a crash, by exit code.
  • Owner setup step 5 (force-pushing main by hand) fires upstream's push:main workflows with no sweep. Say "then dispatch fork-follow once" there.
  • Merge order: this uses ./.github/workflows/fork-release.yml (fork: release workflow for v<upstream>-fork.N on GitHub-hosted runners #2), which calls fork-tests.yml (fork: fast CI on GitHub-hosted runners (fork-tests.yml) #6). Merged before them, the workflow file is invalid, and every run fails at load with no report.

@scuttletoby

scuttletoby commented Nov 12, 2026

Copy link
Copy Markdown
Owner Author

lens: ops+security DI-2 · reviewer: opus high (t3lens-4-di2-r2) · read at: 62216fc
covered: Round 2, fix range 8c32db6..62216fc, all of it: .github/workflows/fork-follow.yml, .github/scripts/fork-follow.sh, fork-follow-push.sh, fork-follow-report.sh, fork-workflows.sh, fork-follow.test.sh. Also read: fork-release.yml on main ee69d98 (all of its inputs, secrets, outputs and publish job), the permissions of fork-tests.yml and release-desktop.yml on main, the triggers of upstream's workflows at dc032d7, and the fork's live settings (environments, workflow states, has_issues, merge methods, branch protection). Design doc read first, all 3 rounds of owner answers included. I formed my view before reading the author's comment 6822304332.
checks:

  • Head: still 62216fc. git range-diff d92e84253d~1..d92e84253d 8c32db60f4~1..8c32db60f4 prints 1: d92e84253 = 1: 8c32db60f. The only tree difference between the two commits is fork: CI check that refuses database schema and migration changes #3's files (fork-schema-check.cjs/.test.cjs, fork-checks.yml). So 8c32db6 is the rebase only.
  • Red→green: bash .github/scripts/fork-follow.test.sh at the head passes 44/44. The same suite with FORK_FOLLOW_DIR/FORK_FOLLOW_WORKFLOW pointed at 8c32db6's script and workflow fails 36 tests. The plan reds are real behaviour: merge commits, failed fetch, unexpected failure, deleted tag, moved tag, advance, stage, none and own_workflows. The push, workflows and report reds come from files that did not exist yet.
  • Every round-1 finding was checked against the code, and each is fixed at its root:
    • DI-1 M1 / DI-2 M3 (merges): refused at fork-follow.sh:118-125, with a backup tag before the move (fork-follow-push.sh:100-103).
    • DI-1 M2 / DI-2 M2 (published but not moved):
      • The report reads tags, branches and releases back (report.sh:42-93).
      • The plan's advance (fork-follow.sh:182-187) compares trees, so a rerun moves main to the existing release and builds nothing.
    • DI-2 M1 (Issues off): has_issues is checked (report.sh:139-143). The summary plus a failing run is the report.
    • DI-2 M4 (upstream workflows): handled by fork-workflows.sh plus four sweeps; see case 5.
    • Minors:
      • fetch, ERR trap and push reasons: fixed.
      • moved or deleted tag: fork-follow.sh:77-94.
      • reusable workflows from the run's commit: the stage action (see minor 1 for the gap).
      • secrets by name, persist-credentials false, token as a GIT_CONFIG header: fixed.
      • report always runs and closes the issue: fixed.
      • cleanup warning: fixed.
      • actions pinned: fixed.
      • previous_tag output: fixed.
  • fork: follow each stable upstream release with the patch series #4's call into fork-release.yml on main (ee69d98):
    • Inputs ref (required string) and publish (boolean) match.
    • The 13 secrets in fork-follow.yml:161-173 match the 13 declared at fork-release.yml:42-68 name for name.
    • The output published is jobs.publish.outputs.published == 'true' (fork-release.yml:74-75), and advance tests the string 'true' (fork-follow.yml:187). That matches.
    • The release job grants actions:read and contents:write. The called jobs ask for at most contents:write (publish, fork-release.yml:467), so the call is valid.
    • The publish guard comes from github.workflow_sha, which is the caller's main.
    • fork: follow each stable upstream release with the patch series #4 merges cleanly into the current main: I merged it locally with no conflicts.
  • Dry runs of the real plan against real upstream history:
  • Hardest cases:
    1. Stop after the main force-push but before the release: this order no longer exists. Main moves only in advance, after published == 'true'.
      • A stop after the release but before the move: the next run plans advance (same tree) and builds nothing.
      • A stop after the move but before the sweep or cleanup: the next run plans none. Its Report sweep catches new workflows, and the leftover candidate branch shows in the table.
      • A rerun of advance: moved=already.
    2. Two runs at once: concurrency fork-follow, no cancel. A queued run checks out main by name and the lease guards the push. After the first run's advance, the queued run plans none. Exception: minor 1.
    3. Conflict: rebase --abort; the report names the patch, the files and the manual commands. Nothing is pushed.
    4. Merge commits: refused and named, the series left as it is. The repo still has allow_merge_commit=true; that is an owner setup step.
    5. New upstream workflows: they cannot be disabled before their first trigger (a GitHub limit).
      • Upstream's push workflows at dc032d7 all filter on branches: main, so the candidate branch push starts none of them.
      • On the main push, a new one starts and is cancelled seconds later by the advance sweep.
      • Existing active ones are disabled by sweep_plan before any push.
      • Gap: a run that finishes before the sweep is not reported (minor 2).
    6. FORK_PUSH_TOKEN and the Environment: see the major.
      • Missing token: refused with the Environment's name.
      • Expired or read-only token: refused by the ls-remote and dry-run push. That was not checked live, but a real push would still fail with git's reason in the report.
    7. Issues disabled: the report summary plus a failing run, with no gh issue call. Live has_issues=false.
    8. Untrusted input: no ${{ }} appears in any run: line.
      • Tags pass the vX.Y.Z / -fork.N regexes before they are used.
      • Commit subjects come only from the fork's own commits and reach scripts through env.
      • Workflow paths from the API and from upstream uses: lines are quoted, except one nit.
      • No upstream code runs in the jobs that hold the token. The plan and advance jobs run only fork scripts, git and node on the fork's .cjs.
  • State question: rerun, half-applied (published but not moved), legacy merges, empty or tagless history, and moved or deleted tags each have a test and give a reason. One gap: a re-run of an old run (minor 1).
  • Live fork right now, not in the diff: no Environment exists; allow_merge_commit=true; main is unprotected; 7 upstream workflows are active. They are deploy-relay, desktop-macos-preview-publish (pull_request_target; it ran with success on fork/ci today), mobile-dev-client, mobile-eas-production, relay-preview, relay-preview-cleanup and web-preview. The first follow runs will refuse until main is rebuilt as v0.1.9 + patches. sweep_plan runs only after a successful plan, so those 7 stay active until then: disable them by hand now.
    finding: major .github/scripts/fork-follow-push.sh:40-51 with fork-follow.yml:54,83-89,181. The Environment rule is not enforced, and its fallback is silent. Today no Environment exists (GET /environments is empty).
  • The first run that names fork-push creates it with no deployment-branch rule.
  • In a job with an Environment, secrets.FORK_PUSH_TOKEN falls back to a repository secret of the same name. So if the token is (or stays) a repository secret, check passes, and nothing tells the owner the token is still readable by a workflow file on any pushed branch. That includes upstream files on fork-follow/candidate.
  • If the token is added to the auto-created Environment without a branch rule, any branch's workflow that names environment: fork-push gets it.
  • The header (fork-follow.yml:23-30) promises "no other job sees the secret", but nothing checks it.
  • Fix:
    • In plan, read repos/$REPO/environments/fork-push with github.token (public), and refuse unless deployment_branch_policy allows only SERIES_BRANCH.
    • In a job without an Environment (e.g. Report), pass HAS_REPO_TOKEN: ${{ secrets.FORK_PUSH_TOKEN != '' }} and refuse or name it when it is true.
      (high on the mechanism, which is GitHub's documented precedence plus the empty environments list; medium on severity: today only people with write access and upstream workflow files on the candidate branch could read it)
      finding: minor .github/scripts/fork-follow.sh:141-158. The release path skips the reusable-workflow comparison that the rebase path does (fork-follow.sh:190-221).
  • Repro: series v0.1.9+patches, RUN_SHA set to a commit whose release-desktop.yml differs. Result: action=release, no stage. The same run against a newer tag gives stage.
  • Real trigger: a person resolves a conflict by hand and force-pushes main onto v0.1.10, then clicks "Re-run all jobs" on the failed run instead of waiting.
    • A re-run keeps the old github.sha, so fork-release.yml and release-desktop.yml come from the v0.1.9 series.
    • The checkout is main by name, so the code built is the v0.1.10 series.
    • The result: a mixed build is published to updaters.
  • Fix: run called() whenever run_sha != head, not only after a rebase, and stage or refuse.
    (high on the trace; low-medium on likelihood)
    finding: minor .github/scripts/fork-workflows.sh:44-45. select(.status != "completed") hides upstream runs that already finished on the pushed SHA before the sweep (a fast job on the main push). Such a run did run with the repository's secrets, and the summary never mentions it. Fix: also list the completed runs of non-own workflows for HEAD_SHA, and report them as "ran before it could be disabled". (high)
    nits (optional):
  • fork-follow.sh:207 for file in $(...) glob-expands paths parsed from upstream uses: lines (a * in a path), so the comparison could miss a file. Use a while read loop.
  • The ERR trap is inherited into command substitutions (set -E). A failing git show | sed inside called() writes a stray "plan stopped at line 201" REPORT while the plan goes on. In every case I traced it is overwritten by the stage message, so this is harmless today.
  • The none path leaves a stale fork-follow/candidate branch in place. The table reports it, but nothing deletes it.

@scuttletoby scuttletoby mentioned this pull request Nov 12, 2026
Open
@scuttletoby

scuttletoby commented Nov 12, 2026

Copy link
Copy Markdown
Owner Author

Fix round 2 · head b898efc76e · fix range 62216fcc84..b898efc76e (1 commit, 58f… amended to b898efc7). No rebase. The branch still sits on #3's a7328e42.

Answers DI-1 r2 (comment 6822548258) and DI-2 r2 (comment 6822734464).

Finding Fix Test (red on 62216fc → green)
DI-2 major / DI-1 minor 3: the Environment's isolation was promised, not enforced New Guard job, with no Environment, runs fork-follow-guard.sh before any job that gets the token, and plan has needs: [guard]. It refuses when secrets.FORK_PUSH_TOKEN != '' there, because outside an Environment that can only be a repository or organisation secret. Only true/false reaches the step. GITHUB_TOKEN has no permission that lists secrets, so the secrets-list API is not usable. It reads environments/fork-push and its deployment-branch-policies with GITHUB_TOKEN (actions: read), and refuses unless there is exactly one rule, branch main, with no pattern and no tag. A missing Environment, all branches, and "Protected branches only" are refused too: the API cannot prove that only main is protected. Every refusal names the Settings path and what to change. 10 guard: cases + 2 workflow: static cases (guard job has no Environment and sees only != ''; plan needs guard) + report: the guard refused
Minor (both): release path skipped the release-workflow comparison changed_release_workflows runs on the release path too, and gives stage with the same command. A run commit missing from the fork is refused ("start a new run"). plan: release path, the run's commit has other release workflows -> stage (DI-1's repro: series on v0.1.11, RUN_SHA on v0.1.10), plan: release path, the run's own commit -> release, plan: the run's commit is not in the fork
Minor (both): sweep blind spots (a) It names upstream .yml/.yaml files in the tree (contents API at the pushed ref) that GitHub does not list, as unregistered= and ::warning::. (b) It cancels unfinished upstream runs from any event and commit, with one listing per unfinished status. (c) It names upstream runs that finished since the Guard job started (SINCE), with event and conclusion, as finished=, leaving out cancelled ones. The report shows (a) and (c) as rows. workflows: an upstream run from another event and commit (pull_request_target) is cancelled too, … finished before the sweep are named …, … files GitHub does not list yet are named …, report: names upstream runs that finished first and files GitHub does not list yet. The fake gh now filters runs by head_sha/status as GitHub does.
Nit: schema-check crash reported as a schema change Node exits 1 for both a refusal and a crash, so only exit 1 with the refusal line counts as a schema change. Anything else gives "The schema check crashed (exit N)". plan: a crashing schema check is reported as a crash… (red); plan: a schema check refusal is reported as a schema change (green on both, regression guard)
Nit: owner step after a manual push The conflict advice and owner setup step 6 both say to start fork-follow.yml once right after the manual push. plan: the conflict advice says to start the follow workflow once…
Nit: for file in $(…) glob Replaced with a while read loop. (by construction; no meaningful red case)
Nit: stray ERR-trap report on_error reports only from the top-level shell (BASH_SUBSHELL == 0), and git show inside called() is `
Nit: leftover candidate branch Action none runs fork-follow-push.sh cleanup. Nothing waits for a candidate when the head is already released. workflow: action none deletes a leftover candidate branch
DI-1 minor 4 (auto-start stage) Left as is, by the brief. The PR body lists it as a follow-up. n/a

Tests.

  • bash .github/scripts/fork-follow.test.sh on b898efc7: 70/70 ok.
  • The same suite with FORK_FOLLOW_DIR/FORK_FOLLOW_WORKFLOW pointed at 62216fcc's scripts and workflow: 24 failed, exactly the 24 new cases above. The 46 that pass there are the earlier cases plus the schema-refusal regression guard.
  • actionlint 1.7.7 with shellcheck 0.10.0 passes on fork-follow.yml placed next to main's fork-release.yml (alone it only reports that fork-release.yml is missing).
  • shellcheck -S warning is clean on the four scripts.
  • Checked read-only against GitHub: the guard against this fork says "Environment fork-push does not exist". Against upstream's Production (rules: branch main + tag v*) it refuses and lists both rules. The contents listing and the status= and created=%3E%3D… runs queries return what the scripts parse.

State question. What happens when this runs again, stops half-way, or meets existing, old or empty state?

Case Behaviour Test
Today's state: repository secret FORK_PUSH_TOKEN exists, no Environment Guard refuses: "delete it under … Repository secrets". The plan never runs, so no empty Environment is auto-created. guard: a repository or organisation secret …
Environment auto-created earlier with no rule Refused: "Any branch may deploy …". guard: any branch may deploy …
Environment with extra or pattern rules, or no rule Refused, with the current rules listed. guard: an extra tag rule, a pattern rule (main*), no rule at all
GitHub API down during the guard Refused with gh's error. The token is not used. guard: the Environment cannot be read
Re-run of an older run after a later run advanced (same base) stage instead of a mixed build. plan: release path, the run's commit has other release workflows -> stage
Re-run whose commit was force-pushed away and is not fetchable Refused: "start a new run". plan: the run's commit is not in the fork
Stopped after a candidate push and before the release, then the head gets released by hand The next run plans none and deletes the leftover candidate branch. workflow: action none deletes a leftover candidate branch + push: cleanup deletes the candidate branch
Manual conflict fix pushed to main The advice says to start the workflow, which sweeps the upstream runs that push started (any event) and names the finished ones. plan: the conflict advice …, workflows: … cancelled too, … finished … are named
A new upstream workflow that no event has reached (empty registration) Named as "not listed yet, so not disabled", not reported as disabled. workflows: upstream workflow files GitHub does not list yet are named
The schema check script missing or broken (old or half-applied tree) "The schema check crashed (exit N)", and the series is left as it is. plan: a crashing schema check is reported as a crash
Round-1 rows (rerun after move, published-not-moved, legacy merges, tagless history, moved or deleted tags, Issues off) Unchanged. Still green (46 earlier cases)

Unproven.

  • No GitHub run yet (needs fork: release workflow for v<upstream>-fork.N on GitHub-hosted runners #2 and fork: fast CI on GitHub-hosted runners (fork-tests.yml) #6 on main).
  • Organisation-secret detection through secrets.X != '' is GitHub's documented precedence. Not live-tested: this repository is not in an organisation.
  • The guard does not check can_admins_bypass (whether an admin's run from another branch passes the branch rule).
  • SINCE misses upstream runs that finished between two follow runs, before this run began. Only runs since the Guard job started are listed.
  • The earlier round's open items stand: ls-remote with an expired token on a public repo, and whether push --dry-run asks receive-pack for auth.

Merge order unchanged: #2 and #6 first.

@scuttletoby scuttletoby mentioned this pull request Nov 12, 2026
Merged
@scuttletoby

scuttletoby commented Nov 12, 2026

Copy link
Copy Markdown
Owner Author

lens: ops+security DI-1 r3 · reviewer: opus high (t3lens-4-di1-r3) · read at: b898efc

covered: full fix diff 62216fc..b898efc (fork-follow.yml, fork-follow-guard.sh, fork-follow.sh, fork-workflows.sh, fork-follow-report.sh, fork-follow.test.sh) plus what it calls (fork-follow-push.sh check/candidate/cleanup, fork-schema-check.cjs exit codes). Every FORK_PUSH_TOKEN reference across .github/workflows (only plan and advance, both environment: fork-push; release passes signing secrets by name, no inherit). Repo state read-only: no Environments exist yet; a repository secret named FORK_PUSH_TOKEN exists (name only, value not read).

checks:

  • Suite: fork-follow.test.sh green at 62216fc (44 ok) and b898efc (70 ok); fork-schema-check.test.cjs 25 pass. Broke it by hand, each went red: guard accepts "protected" -> 1 FAIL; guard ignores REPO_SECRET_PRESENT=true -> 1 FAIL; plan without needs: [guard] -> 1 FAIL; rule match as prefix (main*) -> 2 FAIL. Restored.
  • Round-2 MAJOR fixed at the root for fresh runs: guard has no environment, so secrets.FORK_PUSH_TOKEN there is only repo/org (env > repo > org precedence makes a repo secret a silent fallback in plan; the guard refuses it). Only the boolean reaches the step. Cases where a repo secret exists but reads empty (fork PR, Dependabot) cannot occur on schedule/workflow_dispatch; secret names are case-insensitive so a lowercase twin is caught too. Environment: 404 -> refused with how-to; policy null (auto-created) -> refused; protected-only -> refused; custom must be exactly one rule "branch "; tag rules, patterns, zero rules, extra rules refused. All refusals write $REPORT and exit 1; report job says "the guard refused, so no job got FORK_PUSH_TOKEN".
  • needs chain: plan needs guard; advance needs plan (job-level if requires plan success); release has implicit success(). Guard failure -> plan skipped -> its Environment is not touched/auto-created. workflow_dispatch on a non-main ref: GitHub's environment rule refuses plan/advance (the guard proves that rule is "branch main"). Nothing untrusted in any run: line (all via env:). Guard permissions actions: read, contents: read.
  • Minors fixed: release path now runs changed_release_workflows -> stage; fetch of an unknown RUN_SHA fails closed; schema crash vs refusal told apart (exit 1 + refusal text); sweep now cancels unfinished runs of any event/commit, names finished runs since the guard started and unregistered files; ERR trap only in the top shell; leftover candidate deleted on action none (a staged release candidate never meets "none", since its head is unreleased); for-glob replaced with read loop.
  • State question. First run after merge: no Environment + repo secret present -> guard refuses on the secret first (clear "delete it ... keep it only as an Environment secret"), then on the missing Environment; nothing auto-created. Current repo secret still present -> refused every run until deleted. Two runs at once: workflow-level concurrency, second waits, its guard runs after the first ends. Re-run of an old run: the release path now stages instead of building with old release steps. Re-run of failed jobs / a single job: the guard is NOT re-run (see finding 1). Stops half-way: candidate pushed but no release -> next run rebases/stages again (force-push over the branch); release published but advance failed -> next run plans "advance"; advance refuses a moved branch.

finding: minor .github/workflows/fork-follow.yml:100 "Re-run failed jobs" (or re-run of Plan/Move alone) reuses the guard's earlier success: guard ok, plan fails (say an expired token); the owner then deletes the Environment or adds a repo secret, and re-runs failed jobs days later (allowed for 30 days) -> plan starts without a re-check, GitHub auto-creates fork-push with no branch rule, and the token falls back to the repo secret, the exact state the guard exists to refuse. The next fresh run catches it, so this is a gap in one run, not a lasting hole. Fix: guard outputs attempt=${{ github.run_attempt }}; plan and advance get needs: guard and a job-level if: needs.guard.outputs.attempt == github.run_attempt (job-level, so the skipped job never creates the Environment); add a workflow-text test. (high confidence on the GitHub semantics; low likelihood)
finding: minor .github/workflows/fork-follow.yml:261 (pre-existing, outside the fix diff) The Environment binds the token to the branch, not to fork-follow.yml: any workflow file on main may say environment: fork-push. sweep_push can report a new upstream workflow as "unregistered" (registration lags the candidate push), and advance still pushes main; that file's push run on main then starts before the advance sweep (line 274) can disable it, and could ask for fork-push and get a Contents+Workflows write token (more than the repo secrets any main workflow already sees: it can rewrite fork-follow.yml). This needs a hostile or careless upstream file. Fix: sweep before the advance push, and refuse (or stage for a human) while any upstream workflow file in the candidate is unregistered or active. (medium confidence)

@scuttletoby

scuttletoby commented Nov 12, 2026

Copy link
Copy Markdown
Owner Author

lens: ops+security DI-2 r3 · reviewer: opus high (t3lens-4-di2-r3) · read at: b898efc

covered: the full fix diff 62216fc..b898efc (fork-follow-guard.sh, fork-follow.yml, fork-follow.sh, fork-workflows.sh, fork-follow-report.sh, fork-follow.test.sh) plus what it calls: fork-follow-push.sh check/candidate/advance/cleanup, fork-schema-check.cjs (only node: builtins, runs no upstream code). Ran bash .github/scripts/fork-follow.test.sh and node --test fork-schema-check.test.cjs at both commits: 62216fc has 44 ok + 25 pass, b898efc has 70 ok + 25 pass, all green. Broke it by hand, one change at a time: guard accepts REPO_SECRET_PRESENT=true -> red; plan without needs: [guard] -> red; "protected branches only" accepted -> red; rule check loosened to "contains branch main" -> red; presence hard-coded to false -> red; guard job given environment: fork-push -> red (2 checks). Two changes stayed green (finding 3). Live state, read-only: repo secret FORK_PUSH_TOKEN exists, no Environments, no runs of fork-follow.yml, 17 workflows registered.

checks:

  • Guard before every job that gets the token: yes. plan needs guard and has no if:, so a failed, cancelled or skipped guard skips it. release has a plain if: (implied success()), so it skips too. advance requires needs.plan.result == 'success'. report has no Environment and never reads the secret. On a run from another ref (workflow_dispatch --ref x, a tag), the guard passes, because what it checks does not depend on the ref, but GitHub's Environment rule refuses the plan job. So it fails closed, and the rule, not the guard, is what does the enforcing. Not covered: re-running failed jobs or a single job (finding 1).
  • The presence test: a job with no environment: sees only repository and organisation secrets, and an Environment secret takes precedence over both in an Environment job. So "true" here means a fallback copy exists, and "false" means none this repo can read. I found nothing that makes the secret look empty while it is readable: no Dependabot or fork-PR triggers, the owner is a User (no org secrets), and secrets cannot have empty values. Only the literal true or false reaches the script, and anything else is refused.
  • Environment checks: null policy, protected, unknown, 404 and any other gh error all die, writing a report and ::error::. The rule must be exactly one line, "branch ": a tag rule, a pattern, two rules or no rule is refused, and pagination is handled. Environment names are case-insensitive on both sides.
  • Fail-closed messages: every refusal names the setting to change. A refusal gives report ACTION='', with "None: the guard refused, so no job got FORK_PUSH_TOKEN"; the run goes red and the issue opens.
  • run: lines: no ${{ }} in any run: block. Every input arrives via env:. API strings (rule names, run paths, events) are only printed and never evaluated.
  • Permissions: guard has actions: read and contents: read, plan has actions: write (to disable workflows), and the token reaches only the push/advance/cleanup steps. All checkouts use persist-credentials: false, and actions are pinned by SHA.
  • Round-2 items are fixed at the root. The token isolation check is the guard job. The release path now runs changed_release_workflows too, so a re-run of an old commit stages instead of building with old steps. The sweep now cancels unfinished runs of every event and commit, names runs that finished since the guard started, and names unregistered files. Schema check: a crash vs a refusal is told apart by exit status plus the refusal text. Also done: a dispatch hint after a manual push, the for-glob replaced by while read, the ERR trap limited to BASH_SUBSHELL 0, and the leftover candidate deleted on action none.
  • State question (run again / stop half-way / existing, old or empty state):
    • First run after merge, with the current repo secret still present: the guard refuses on the secret. Once the owner deletes it, the next run refuses on the missing Environment (404). Neither run reaches an environment: job, so GitHub never auto-creates fork-push without a rule. Until the owner moves the token, though, any workflow file on any pushed branch can read the repo secret today, whether or not this PR merges. It needs moving now, not only before the first run.
    • Re-run of an old failed run: run_sha differs from the candidate's release workflows, so the run stages and builds nothing. If the commit is gone, the run says to start a new run. The guard verdict, however, is reused (finding 1).
    • Stopped half-way: the candidate push is a force-push to fork-follow/candidate. advance uses a lease, and a re-run finds the move already done. A crash in the sweep leaves workflows disabled but loses the listing (outputs are written only at the end); the run still goes red via SWEEP_RESULT.
    • Two runs at once: the concurrency group serialises them. The sweep skips own workflows, so it does not cancel the queued follow run or a manual fork-release run.
    • Empty history (no tags) is covered by an existing test.

finding: minor .github/workflows/fork-follow.yml:100 - "Re-run failed jobs" or "Re-run job" after the plan or advance failed reruns plan, release and advance, but reuses the guard's success from attempt 1. A repo secret re-added, or the rule widened, since then, perhaps days later, is not seen, and plan or advance gets the token anyway. The same applies to advance, which gets the token after release, often an hour or more after the guard looked. Impact: the alarm comes one run late; the token's exposure itself is decided by GitHub's settings either way. Fix: add attempt=$GITHUB_RUN_ATTEMPT to the guard's outputs, with a first step in plan and advance that refuses unless needs.guard.outputs.attempt == github.run_attempt ("re-run all jobs"). Optionally also repeat check 2 (the Environment rule) there. (high)

finding: minor .github/scripts/fork-workflows.sh:83 - in this repo today, 12 upstream files on main are not registered: desktop-macos-preview, issue-labels, mobile-android-apk, mobile-eas-preview-fork, mobile-fingerprint-label, mobile-showcase-screenshots, publish-aur, release-desktop, release, stale-needs-more-info, thread-transfer-report and windows-tests. Several of them can never be registered by an event the fork sends: workflow_call-only (release-desktop, publish-aur), workflow_dispatch-only (windows-tests, mobile-showcase-screenshots) and workflow_run-only. So every run, green ones included, prints the bold row "not disabled (the sweep after their first run disables them)" with about 12 files, and a ::warning::. For these files that promise is untrue, and a genuinely new push-, pull_request_target- or schedule-triggered upstream file, the case the row exists for, is buried in permanent noise. Fix: name only files new since the series' previous base, i.e. git diff --diff-filter=A previous_head..TREE_REF, or only files whose on: can fire in the fork. Computed read-only against the live repo. (high)

finding: minor .github/scripts/fork-follow.test.sh:546 - adding "if: always()" (or "!cancelled()") to the plan job keeps the suite green, yet it lets the first job with the token run after the guard refused. The check greps only the needs: line. Likewise, deleting the SERIES_BRANCH wildcard refusal at fork-follow-guard.sh:81 stays green, because no test sets FORK_SERIES_BRANCH to a pattern. Fix: assert that the plan block has no if: (or only success()-implying ones), and add a guard case with SERIES_BRANCH='main*' and rule 'branch main*'. Both mutations were run by hand. (high)

@scuttletoby

scuttletoby commented Nov 12, 2026

Copy link
Copy Markdown
Owner Author

fix round 3 · head fc08057 · range b898efc..fc08057 (1 commit) · answers DI-1 r3 (6823109924) and DI-2 r3 (6823151991)

Fixed

  1. (DI-1 + DI-2) A guard result reused by "Re-run failed jobs" or "Re-run job". The guard outputs attempt, set by a step after the guard script. plan has if: needs.guard.outputs.attempt == github.run_attempt, a job-level if: with the implied success(), so a skipped job never reaches the Environment. advance now needs: [guard, plan, release] with the same condition. The report says "Re-run all jobs" when a token job was skipped for this reason. Right before the push, advance also runs fork-follow-guard.sh --environment-only (check 2 again; check 1 cannot be made inside the Environment).
  2. (DI-1) The Environment binds the token to the branch, not to the file. The plan outputs new_workflows: upstream files added between the old and the new base (--no-renames, so a rename counts as new). In advance, a sweep runs before the push with REQUIRE_NEW_OFF=true. It disables the listed upstream workflows, then refuses while any of new_workflows is in the candidate tree and unlisted. The reason names the files and gives the person the manual move command, or a fork patch that deletes them. The move step has no if:, so it is skipped, and so is the candidate cleanup. The later sweep still runs.
  3. (DI-2) Permanent warning noise. Only files in new_workflows count as unregistered (a warning and a bold row). The others go to unregistered_old, shown in a collapsed <details> note with no warning.
  4. (DI-2) Test gaps. The plan header must have exactly one job-level if:, the attempt check, with no always(), cancelled() or failure(). There is now a guard case with SERIES_BRANCH='main*' and the rule branch main*.

Tests. fork-follow.test.sh: 87 ok on fc08057. The new test file run against b898efc's scripts and workflow (FORK_FOLLOW_DIR/FORK_FOLLOW_WORKFLOW): 14 FAIL (plan new_workflows ×2, guard --environment-only, sweep new/old ×2, refusal before the move, report collapsed note, report stale guard ×2, workflow: advance needs guard, plan only-if, advance attempt, attempt output after the guard, checks before the move). fork-schema-check.test.cjs: 25 pass. actionlint 1.7.7 with shellcheck 0.10.0: clean, next to main's fork-release.yml/fork-tests.yml.
Mutations, one at a time, each red: plan if: always(); if: always() && <attempt>; if: ${{ !cancelled() }}; the plan's attempt check removed; advance's attempt check removed; the guard's wildcard refusal removed (fork-follow-guard.sh *[][*?]*); the advance re-check removed; REQUIRE_NEW_OFF: "false"; the sweep's refusal disabled; attempt taken from github.run_attempt instead of the step after the guard; --no-renames dropped.

State question: what happens when this runs again, stops half-way, or meets existing, old or empty state?

Case What happens Test
Re-run failed jobs after plan failed (guard passed in attempt 1) plan is skipped (attempt 1 ≠ 2) and never reaches fork-push. The report fails: "did not run in this attempt … Re-run all jobs" report: guard passed in an earlier attempt, the plan skipped; workflow: plan only-if
Re-run failed jobs after the release failed release re-runs (it has no token), advance is skipped. The report fails with the stale-guard reason. The next fresh run plans advance report: released, but the move skipped …; workflow: advance needs this attempt's guard
Re-run of the report alone after a good run no token job was skipped for it, so no alarm report: re-run of the report alone … does not cry wolf
Guard refuses, then "Re-run failed jobs" the guard re-runs in the new attempt, so attempt matches when it passes existing guard cases (attempt is set only after the script: workflow test)
Environment rule widened during a long release advance re-checks before the push and refuses (the token is in the job, but nothing is pushed) guard: --environment-only still refuses …
New upstream push:main workflow on the release, not listed after the candidate push advance refuses; main not moved, release kept, candidate kept. Each later run plans advance and refuses again until a person moves main or adds a delete patch workflows: before the move, a new unlisted upstream file refuses it …; plan: names new files (a rename too)
New upstream file that the candidate push did register the sweep before the move disables it, then the move goes on workflows: before the move, a listed new file … do not refuse it
Stops half-way: refused before the move release published, branch at the old head, the post-move sweep still runs. The next run plans advance (release found by tree) plan: a published release of the rebased files exists -> advance (existing)
Existing/old state: about 12 unlisted upstream files from earlier releases named in a collapsed note, no warning, the move is not held up workflows: unlisted file from an earlier release …; report: collapsed note
Same base (release/none path) new_workflows empty, so nothing holds up a move plan: same base -> no new upstream workflow files
Empty state: guard output missing (a guard that failed or was skipped) '' ≠ run_attempt, and success() fails too: plan skipped workflow: plan only-if (mutation always() red)

Unproven

  • No GitHub run. That "Re-run failed jobs" keeps the outputs of jobs it does not re-run and raises github.run_attempt comes from GitHub's documentation and DI-1/DI-2, not from a run.
  • Whether the push of fork-follow/candidate registers a new on: push: branches: [main] file. Probably not, so such a release needs a person every time (by design here).
  • A file upstream changed (not added) that stays unlisted, for example a workflow_call-only file that gains a push trigger and environment: fork-push, does not hold up the move. This is outside the brief's "new since the last base" scope.
  • The re-check before the push runs scripts from the candidate checkout. A candidate without fork-follow-guard.sh (a release from before this PR) fails that step closed.

@scuttletoby

scuttletoby commented Nov 12, 2026

Copy link
Copy Markdown
Owner Author

lens: quick check r4 · reviewer: opus high (t3lens-4-r4) · read at: fc08057

covered: the full fix diff b898efc..fc08057 (fork-follow.yml, fork-follow-guard.sh, fork-workflows.sh, fork-follow.sh, fork-follow-report.sh, fork-follow.test.sh) and what it calls into: the plan's new_workflows path (base -> target, also on action advance), the sweep's listed/contents queries, the guard's Environment and rules reads, and fork-follow-push.sh as reached from advance. bash .github/scripts/fork-follow.test.sh: 87/87 at both b898efc and fc08057. node --test .github/scripts/fork-schema-check.test.cjs: 25/25 (node 24.21). 20 hand mutations: 17 went red, 3 survived (finding 3).

checks:

  • Re-runs. "Re-run failed jobs" or "Re-run job" on plan or advance keeps the guard's output attempt=N while github.run_attempt is N+1, so plan and advance skip at job level and the Environment is never reached. "Re-run all" runs the guard again and sets a fresh attempt. A refused guard has no attempt output (the passed step runs only after the script), and '' never equals a run_attempt (>= 1). Both sides are strings, and GitHub's == coerces a string/number mix to numbers, so "2" against 2 works either way. advance keeps !cancelled() and states the attempt test explicitly. One thing is not gated, and it is not a token job: a "re-run failed jobs" of a failed release publishes again from attempt 1's plan outputs. advance then skips, and the report says to re-run all.
  • Stops half-way, runs again. If the release is published and the move is refused before the push (Environment re-check or the new-file refusal), the next run plans advance with the same base -> target new_workflows and refuses again until the file is listed or deleted. That is consistent, but it can repeat daily if the candidate push never registers the file (the PR lists this as not tested). Report alone, run again after a good run: passes. A queued or re-run older commit uses its own workflow file, so the old semantics apply as a whole.
  • Old or empty state. On the same base new_workflows= is empty, so there is no refusal (correct: main already has the files). If NEW_WORKFLOWS is unset, every unlisted file counts as old: no warning, no refusal. Empty plan list: only reached by those paths. A gh error or 404 on the workflows list or the contents listing goes to die, so it fails closed. The workflows list is paginated, and the contents API needs no pagination at this size.
  • Renames: --no-renames counts them as A (tested). A file deleted and re-added in one release range shows as M or no change, so it falls under finding 2. Deleted in release N and re-added in N+1 shows as A, so it is caught.
  • Candidate checkout in advance: guard --environment-only, fork-workflows.sh and fork-follow-push.sh run from the candidate. None of them sources a file, calls a tree-local binary or runs node or pnpm. They call only gh, git, jq and awk from PATH. An upstream release changes these files only by adding the same path, and the replay then hits an add/add conflict and refuses. The exception is an identical copy, where the fork patch is dropped and a later upstream edit then lands unopposed. That is negligible, and the push step already had this trust before this round. A sparse checkout of .github/scripts at github.sha, like the report job uses, would remove it entirely. Not a finding.
  • run: lines: none of the new ones interpolates ${{ }}. Values reach the scripts through env (GITHUB_RUN_ATTEMPT and plan outputs).

finding: minor .github/scripts/fork-follow.sh:159. new_workflows reads git diff --name-only with git's default core.quotepath, so a new upstream file with a non-ASCII name (or a quote, backslash or tab), say .github/workflows/dé.yml, prints as ".github/workflows/d\303\251.yml". The \.ya?ml$ grep then drops it, and NEW_WORKFLOWS is empty. The sweep sees the real path through the contents API, finds it unlisted and files it under unregistered_old, so there is no warning and no refusal: fix (2) fails open for the exact case it targets. Reproduced locally with a two-commit repo: new_workflows=[]. Use git -c core.quotepath=off diff -z --name-only ... with tr '\0' , (or read NUL-separated), and add a test with a non-ASCII name. (confidence high on the mechanism, medium that GitHub runs such a file name)

finding: minor .github/scripts/fork-follow.sh:159. Because of --diff-filter=A, an upstream file that already existed at the old base, is unlisted by GitHub, and is changed on this release (say it gains on: push and environment: fork-push) does not hold up the move. It runs on main with the token in reach. The PR names this under "Not tested". It is cheap to close without bringing back the noise from (3): use --diff-filter=AM (or any path whose blob differs between base and target). That stays scoped to the release, so older untouched unlisted files still collapse quietly. (confidence high)

finding: minor .github/scripts/fork-follow.test.sh:607-613. Three mutations of the fixes leave all 87 tests green:
(a) in advance's if:, needs.guard.outputs.attempt == github.run_attempt || instead of &&. Precedence gives (... && attempt) || (action == advance || rebase && release ok). So "re-run failed jobs" of a failed advance, with plan's kept outputs, runs with attempt 1's guard, which is minor (1) again. The test greps for the substring only.
(b) continue-on-error: true on "Check again that only the series branch may deploy to fork-push".
(c) continue-on-error: true on sweep_before.
In (b) and (c) the refusal becomes a warning and the move still runs. To close them, match the exact line needs.guard.outputs.attempt == github.run_attempt && and refuse continue-on-error in advance_job before the move step. (confidence high)

noreply and others added 11 commits November 12, 2026 13:07
@noreply @claude
fork: follow each stable upstream release with the patch series
048fe93 
fork-follow.yml runs daily. When upstream has a newer stable tag it rebases
the series onto it (patches upstream now has drop out as empty), runs the
schema check, tests, builds and publishes through fork-release.yml, then
moves the series branch with a lease. When the series has commits no fork
release has, it releases them. A conflict, schema change, upstream commits
inside the series, a red test or a failed build open or update one issue and
publish nothing. Pushing a rebase needs a FORK_PUSH_TOKEN secret, because
GITHUB_TOKEN may not push commits that change workflow files.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@noreply @claude
fork-follow: report from real state, guarded pushes, upstream workflo…
58e9979 
…ws off

Review round 1 on the follow-upstream workflow.

- The plan picks one of five actions: none, release, rebase, advance
  (the rebased series is already released; only move main) and stage
  (the rebase changes the release workflow itself, so the candidate is
  pushed and a person starts its release).
- Merge commits on main are refused instead of being flattened by a
  rebase; deleted or moved upstream tags and failed fetches give a
  reason and the right advice.
- All pushes go through fork-follow-push.sh with FORK_PUSH_TOKEN only.
  Every run checks the token first (missing, refused, no push access).
  Moving main uses a lease, treats "already there" as success, and
  first keeps the old series as tag fork-follow/pre-<tag>-<sha12>.
- fork-workflows.sh disables upstream's workflows and cancels their
  runs after each push and at the end of the run.
- The report job runs on every run and reads branches, tags and
  releases back from the repository instead of guessing from job
  results. Its summary is the report; an issue is opened, commented
  on or closed only when the repository has Issues enabled.
- fork-follow.test.sh covers all of this against local git remotes and
  a fake gh, plus static checks on the workflow file (secrets passed by
  name, token only in push steps, persist-credentials false, pinned
  actions). The plan job runs it first.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@noreply @claude
fork-follow: the pushing jobs run in the Environment fork-push
63b604d 
A repository secret can be read by a workflow file on any pushed branch.
An Environment secret limited to the series branch reaches only runs from
that branch, so the plan and advance jobs (the only ones that read
FORK_PUSH_TOKEN) now run in the Environment fork-push.

When the token is empty there, the check fails and its reason (in the
run's summary) names the Environment, how to limit it to the series
branch, and why the token should not stay a repository secret. A test
checks that exactly the jobs that read the token use the Environment.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@noreply @claude
fork-follow tests: tagless history and a refused candidate push
b660f93 
Pin down two state cases from review round 1: upstream without a stable
tag, or a series that contains none, is refused with a reason and plans
nothing; a refused candidate push leaves git's reason in the report.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@noreply @claude
fork-follow: guard the token's isolation, compare release workflows o…
8b641df 
…n every path

The Guard job (no Environment) runs before any job gets FORK_PUSH_TOKEN and
refuses unless the repository holds no repository or organisation secret of
that name (it sees only whether `secrets.FORK_PUSH_TOKEN` is empty there) and
the Environment fork-push exists with exactly one deployment rule: the
branch main. Each refusal says what to change.

The plan compares the release workflows of the run's commit with the
candidate's on the release path too, so a re-run or queued run of an older
commit stages instead of building with other release steps. A commit missing
from the fork is refused.

The sweep cancels unfinished upstream runs from any event, names upstream
runs that finished since the run began, and names upstream workflow files
GitHub does not list yet. The report shows both.

Nits: a crashing schema check is reported as a crash; the conflict advice
says to start the follow workflow after the manual push; `while read`
instead of an unquoted `for`; the ERR trap reports only from the main shell;
action none deletes a leftover candidate branch.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@noreply @claude
fork-follow: guard per run attempt, refuse the branch move while a ne…
70b9612 
…w upstream workflow is unlisted

"Re-run failed jobs" and "Re-run job" keep the Guard job's result from an
earlier attempt without running it again. The guard now outputs the attempt it
passed in (set only after the guard script), and plan and advance run only
when that is this attempt: a job-level if, so a skipped job never reaches the
Environment. The report says to re-run all jobs. Before the branch move,
advance checks the Environment's rule again (fork-follow-guard.sh
--environment-only), because the release can take an hour or more.

The Environment ties the token to the branch main, not to fork-follow.yml, so
any workflow file on main may ask for it. The plan names the upstream workflow
files that are new on the target release (a rename counts as new). Advance
sweeps before the push, and refuses to move main while one of them is in the
candidate and GitHub does not list it, since it cannot be disabled first. The
reason says how a person moves main, or that a fork patch can delete the files.

The sweep names only the new files as unregistered (with a warning). Unlisted
files from earlier releases, most of which only run on events the fork never
sends, go to unregistered_old and into a collapsed note in the summary.

Tests: the plan's only job-level if is this attempt's guard (always(),
!cancelled() and failure() turn it red); advance needs this attempt's guard; a
pattern series branch with the matching rule is refused; new vs old unlisted
files; the refusal before the move; the report on a stale guard.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@noreply @claude
fork-follow: scripts reach gh only in a run or under the tests' stub
8deff41 
A test run against other scripts made a real gh API sweep and disabled a
workflow of the live fork. Now every script that calls gh sources
fork-gh-guard.sh: outside a GitHub Actions run it refuses unless a test
runs it (FORK_TEST_MODE) with the stub as the gh on PATH, no GitHub token
and a .invalid GH_HOST. It exits before any gh call, and every later gh
call checks again. The tests install the stub through fork-test-gh.sh,
which also empties the gh config, so even a script without the guard
finds nothing to log in to.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@noreply @claude
fork-release: reach gh only in a run or under the tests' stub
51af29d 
fork-release.sh sources fork-gh-guard.sh like the follow scripts, and
fork-release.test.sh installs its stub through fork-test-gh.sh, with tests
that a run without the stub, by hand, with a token or with a real GH_HOST
refuses before any gh call.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@noreply @claude
fork-follow: a fork-*.yml is the fork's own even before main has it
2e10b37 
The sweep took only the files the series adds as the fork's own, so a
fork pull request that adds a fork-*.yml had that workflow disabled (and
its runs cancelled) by the next sweep. Now every
.github/workflows/fork-*.y[a]ml is the fork's own. In return the plan
refuses an upstream release that has such a file and the candidate keeps
it, since the sweep would never disable it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@noreply @claude
fork-follow: changed and non-ASCII upstream workflows hold the move too
717ea02 
new_workflows read `git diff --name-only` with git's quoting, so a new
upstream workflow with a non-ASCII name came out quoted, failed the .yml
filter and slipped past the branch-move refusal. It now reads the diff
NUL-separated with core.quotepath=off, and refuses a name with ',' or a
newline (the lists are ','-joined). It also takes changed files
(--diff-filter=AM): an unlisted upstream workflow that now runs on push
holds the move like a new one. The lists are built in the plan's own
shell, so a refusal stops the plan.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@noreply @claude
fork-follow tests: an || in advance's if:, continue-on-error on a ref…
c3ee354 
…usal

Three breaks left every test green: `||` after the attempt check in
advance's if:, and continue-on-error on the Environment re-check or on
the sweep before the move (the refusal turns into a warning and the
branch moves). The tests now read advance's if: as one conjunction (no
`||` outside parentheses) and allow continue-on-error only in the Report
job.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@noreply @claude
fork-secret-scan: fail when gitleaks logs an error and still exits 0
c861d84 
gitleaks 8.30.1 logs `ERR error="stderr is not empty"` when git writes to stderr while it reads
the log, may drop the rest of the diffs, and exits 0. The scan now fails (2) on any ERR line in
gitleaks' log, before the commit count. Test: a git wrapper that writes a stray stderr line;
with a 3,000-file commit the real gitleaks misses a key in the last file and passes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@scuttletoby

scuttletoby commented Nov 12, 2026

Copy link
Copy Markdown
Owner Author

Fix round 4

New head c861d84, pushed with --force-with-lease=fork/follow-upstream:fc080577.

Rebase

I rebased #4's own six commits onto main ae0d6865e, which now includes #3 and #6, with git rebase --onto origin/main a7328e42. The commits from #3's older head were dropped because main already has them. There were no conflicts.

git range-diff a7328e42..fc080577 origin/main..70b9612ab: all six commits are =. No commit changed beyond moving to the new base.

Fixes: 70b9612ab..c861d8427

Each new test was red on 70b9612ab, the rebased head before the fix commits, and is green now.

# Commit Change Red before, green after
1 8deff417c (fork-follow scripts)
51af29d39 (fork-release, #2's files)		 The scripts that call gh source the new fork-gh-guard.sh. Every gh call goes through a gh() function that checks again each time. gh is allowed only in a GitHub Actions run (GITHUB_ACTIONS=true, not test mode), or in test mode (FORK_TEST_MODE) under these conditions: the gh on PATH is the stub FORK_GH_STUB names, it carries the stub's marker line, no GitHub token is set, and GH_HOST is a .invalid host. Anything else exits 1 before the first gh call. The tests set this up first with fork-test-gh.sh: they unset the tokens, set GH_HOST=gh-stub.invalid, give gh an empty config, put the stub first on PATH, and stop if gh resolves anywhere else. fork-follow.test.sh, 6 tests: gh: a test run without the stub first on PATH fails closed and never calls gh; with GH_TOKEN set; with GITHUB_TOKEN set; a gh named as the stub but without the marker; PATH changed after the start (each call checks again); and run by hand.
fork-release.test.sh, 4 tests: without the stub; with a token set; run by hand (no test mode, not in a run); the per-call re-check.
2 2e10b37a6 In fork-workflows.sh, own() also counts any .github/workflows/fork-*.yml or .yaml as the fork's own, even when main doesn't have the file yet. The plan refuses a release where upstream itself ships a fork-*.yml that the candidate keeps, so upstream can't hide behind the name. workflows: a fork-*.yml/.yaml absent from main (a fork PR adds it) is never disabled, its runs never cancelled
plan: rebasing onto a release with an upstream fork-*.yml is refused and names it
plan: same base with an upstream fork-*.yml is refused too
workflows: an upstream name that only contains 'fork' is still disabled (this one was a control and stayed green)
3, 4 717ea028d fork-follow.sh reads the workflow diff with -z and core.quotepath=off, and refuses names that contain , or a newline. It uses --diff-filter=AM, so a changed upstream workflow holds the move just like a new one. Both lists are built in the main shell, so a refusal stops the plan. The message now says "new or changed on this release". plan: a changed upstream workflow and a non-ASCII new one are both named, unquoted
plan: a new upstream workflow with ',' in its name (cannot be listed) is refused, nothing planned
the matching workflows: sweep test
5 c3ee35461 Two new workflow tests. I checked them against three mutants of fork-follow.yml: one with || in advance's if:, one with continue-on-error on the Environment re-check, and one with it on the refusal step. Each mutant turns a new test red, while the old tests stayed green on all three. workflow: advance's if: is a conjunction (no top-level ||) of the plan's success and this attempt's guard
workflow: continue-on-error appears only in the Report job (never on the guard, the re-check or the sweep before the move)
6 c861d8427 (#6's files, kept as a separate commit) fork-secret-scan.sh fails with exit 2 when gitleaks exits 0 but logs an ERR line, such as git stderr or ERR error="stderr is not empty". This check runs before the commit-count check. The stand-in gitleaks now copies git's stderr the way 8.30.1 does. pull request: gitleaks logging a git error (stderr not empty) and passing fails (red with the stand-in).
pull request: a stray git stderr line that hides a key in a wide commit fails: one commit with 3,000 files and a key in the last one, plus a git wrapper that writes one stray stderr line. Before the fix, real gitleaks 8.30.1 printed "no leaks found" and the scan exited 0. Now it exits 2.
Control: without the stray line the same scan passes (0).

Full fork test set (local, all tokens unset, GH_HOST .invalid)

  • fork-follow.test.sh: 104 ok
  • fork-release.test.sh: 38 ok
  • fork-secret-scan.test.sh: 18 ok with the stand-in, and green with real gitleaks 8.30.1
  • fork-release-workflow.test.py: all passed
  • node --test: fork-schema-check.test.cjs 47/47, fork-ci-tests.test.cjs 8/8, fork-test-shards.test.cjs 4/4, fork-tests-drift.test.cjs 6/6
  • node fork-ci-tests.cjs: 0

None of this PR's scripts were run against the real GitHub API.

CI on c861d84

Not green, and both failures come from main, not from this PR. Checked at 13:3x UTC:

  • Passing: Build and Typecheck, Rust, Release Smoke, Secret scan, Workflow lint and upstream drift (where the fork shell tests run), and Test Server 1 to 8.
  • Still pending: Test 1, Test 2, Test Web 1, Test Web 2.
  • Lint fails: 8 × t3code(no-low-value-comments), all in .github/scripts/fork-schema-check.cjs, which is fork: CI check that refuses database schema and migration changes #3's file. This PR doesn't touch it. Main ae0d6865 fails Lint the same way in its own push run (43223856243).
  • No database schema changes fails in actions/checkout of the default branch, before any script runs: fatal: No url found for submodule path '.repos/alchemy-effect/submodules/distilled' in .gitmodules. Upstream's tree has gitlinks under .repos/alchemy-effect/submodules/ but no .gitmodules. The job is pull_request_target, so it runs main's fork-checks.yml, and a fix in this PR can't reach it. It needs a fix on main, either in fork-checks.yml or in fork: CI check that refuses database schema and migration changes #3's lint comments. I left both alone because they are outside fork: follow each stable upstream release with the patch series #4's files.

State question: what happens on a rerun, a run that stops half-way, or old, existing or empty state?

One test per row.

Case What happens Test
Rerun: advance after it moved It succeeds without pushing; the series is already there. push: advance again after it moved (re-run) succeeds without a push
Rerun: the sweep Workflows that are already disabled are left alone, and finished or cancelled runs aren't touched again. workflows: an active upstream workflow is disabled and named; own, disabled and dynamic ones are left
workflows: upstream runs that finished before the sweep are named …; own and cancelled ones are not
Rerun: the report alone after a good run No false alarm. report: a re-run of the report alone after a good run does not cry wolf
Rerun: the guard passed in an earlier attempt and the plan skipped It fails and says to re-run all jobs. report: the guard passed in an earlier attempt, the plan skipped -> fails, says to re-run all jobs
Rerun: release, re-running failed jobs It refuses to remove drafts for a version whose tag exists. remove-drafts: refuses a version whose tag exists (re-run failed jobs)
Half-applied: published but not moved The report says both, and never says "nothing was published". A later run advances to the published release without building again. report: published but not moved -> says both, never 'nothing was published'
plan: a published release of the rebased files exists -> advance to it, build nothing
Half-applied: a failed release with no tag The report says "none published", read from the repository. report: a failed release with no tag -> 'none published', read from the repository
Old state: an older test file or a run by hand The scripts refuse to start without test mode and a real Actions run, so an old test file that runs these scripts can no longer reach gh. gh: run by hand (no test mode, not in a GitHub Actions run) refuses
gh: a test run without the stub first on PATH fails closed and never calls gh
Existing state: a fork-*.yml only on a PR It is never disabled and its runs are never cancelled. workflows: a fork-*.yml/.yaml absent from main (a fork PR adds it) is never disabled, its runs never cancelled
Old state: unlisted upstream files from earlier releases They go in a collapsed note and don't refuse the move. report: unlisted upstream files from earlier releases go in a collapsed note, not a bold row, and pass
workflows: before the move, a new upstream file GitHub lists (now disabled) and older unlisted ones do not refuse it
Old state: a stray higher tag or an older ref Refused. resolve: refuses a stray higher tag on an old commit
resolve: refuses a build below a published release
Empty: tagless upstream history Refused with a reason; nothing is planned. plan: upstream has no stable tag -> refused with a reason, nothing planned
Empty: an empty own list Nothing is disabled; it refuses to guess. workflows: an empty own list disables nothing (refusing to guess)
Empty: a failed workflow listing It fails with a reason. workflows: a failed listing fails with a reason

Not proven

  • Old test file plus old scripts. A checkout from before this round, running its own test file against its own scripts, has neither guard. Nothing in this PR can change an older checkout. Since this round, the scripts refuse unless test mode is set up, and the tests set up the stub, the .invalid host and the empty config before any script runs.
  • The guard only checks the run's environment. It can't tell a real fork workflow from any other GitHub Actions run of these scripts, so it relies on GITHUB_ACTIONS=true.
  • Fix 6 is tested only against gitleaks 8.30.1. The ERR match assumes its log format, with or without a timestamp before ERR. Another version could log errors differently.
  • The mutants for fix 5 were checked locally. They are not kept as a test.

@scuttletoby scuttletoby mentioned this pull request Nov 12, 2026
Merged
@scuttletoby

scuttletoby commented Nov 12, 2026

Copy link
Copy Markdown
Owner Author

lens: correctness+security r5 · reviewer: opus high (t3lens-4-r5) · read at: c861d84

covered: the rebase (range-diff, and whether the dropped commits are in main); the full fix diff 70b9612..c861d84; everything it calls into: fork-gh-guard.sh, fork-test-gh.sh, every gh call in fork-workflows.sh, fork-follow-guard.sh, fork-follow-report.sh and fork-release.sh, fork-follow-push.sh (git only, no gh), the workflow_paths/fork_named/refuse_upstream_fork_named functions in fork-follow.sh, own() in fork-workflows.sh, the ERR check in fork-secret-scan.sh, and which test steps in fork-follow.yml, fork-release.yml and fork-tests.yml run with a token.

checks:

  • Rebase: git range-diff a7328e4..fc08057 origin/main..70b9612 shows all six commits as "=". The dropped commits are 2146267, e9afd0c and a7328e4 (fork: CI check that refuses database schema and migration changes #3's older head). Main has them as squash ae0d686, in fork: CI check that refuses database schema and migration changes #3's later, larger final form: fork-checks.yml is byte-identical, and the two .cjs files are the extended versions. fork: follow each stable upstream release with the patch series #4 touches none of these 3 files. Upstream base 24e7747 is an ancestor of main.
  • (a) gh isolation. Every gh call in the 4 scripts goes through the guard's gh() function. None goes through xargs, env or command. fork-follow.sh and fork-follow-push.sh mention gh only in messages. The test steps (fork-follow.yml:147, fork-release.yml:108, fork-tests.yml:224) have no GH_TOKEN, and the checkouts use persist-credentials: false. fork_test_gh unsets the tokens and GITHUB_ACTIONS, sets an empty GH_CONFIG_DIR and a .invalid GH_HOST, puts the stub first on PATH and stops if gh resolves anywhere else. So even an older script with no guard, run from this test file, reaches only the stub. An older test file against the new scripts fails closed locally, because there is no FORK_TEST_MODE and no GITHUB_ACTIONS. GH_TOKEN set in the caller's environment: the guard refuses, and fork_test_gh unsets it anyway. Remaining gaps are under the minor finding below.
  • (b) Removing the fork-* rule from own() turns 2 tests red. Removing the two refuse_upstream_fork_named calls turns the 2 plan refusal tests red. Existing state: the upstream tree at 24e7747 has no .github/workflows/fork-*.yml (mobile-eas-preview-fork.yml does not match), so the new same-base refusal does not trip the next run on today's state.
  • (c) Changing --diff-filter=AM back to A turns "plan: a changed upstream workflow and a non-ASCII new one ..." red. Dropping -z and core.quotepath=off turns the same test red. Reverting all of 717ea02 also turns the ',' refusal and the unlisted-file workflows test red. Empty diff: new_list and own_list come out empty, which is fine under set -u. The ',' and newline refusal runs in the plan's own shell, so fail exits the plan.
  • (d) Mutants I broke by hand in fork-follow.yml: a top-level "always() ||" in advance's if: (2 tests red); "(attempt check || true)" inside the parentheses (red); continue-on-error on the guard step, on the re-check and on sweep_before (red each). The 3 tests do turn red. See the finding for what they do not catch.
  • (e) The ERR line now gives exit 2. With real gitleaks 8.30.1: at head, all 18 tests pass. With c861d84's script change reverted, "a stray git stderr line that hides a key in a wide commit fails" fails with exit 0, expected 2. With the stand-in and the change reverted, the stderr-not-empty test fails the same way.
  • Test runs in a scratch clone at c861d84, with GH_TOKEN, GITHUB_TOKEN and GITHUB_ACTIONS unset and GH_HOST=bogus.invalid: fork-follow 104 ok, fork-release 38 ok, fork-secret-scan 18 ok (stand-in and real 8.30.1), fork-release-workflow.test.py ok, node --test on schema-check, ci-tests, test-shards and tests-drift all 0, fork-ci-tests.cjs 0. Each fix reverted on its own turns its tests red: the guard source lines removed from the 3 follow scripts (6 gh tests red), from fork-release.sh (4 red), own(), the refusal calls, AM, -z, and c861.
  • State question (rerun, half-applied, old, existing, empty): I checked the author's table in 6824882575 against the tests; each row has a test, and they pass. workflow_paths overwrites $logs/paths on each call, so a rerun or a second call in the same job is safe. The same-base refusal runs on every rerun, but today's upstream tree has no fork-*.yml, so it doesn't fire. Empty upstream workflow diffs give empty lists, not errors.
  • CI on c861d84: the only reds are Lint (8x no-low-value-comments, all in .github/scripts/fork-schema-check.cjs, which is fork: CI check that refuses database schema and migration changes #3's file), "No database schema changes" (main's checkout/submodule problem), and fork-ci-result (which only rolls them up). fork: follow each stable upstream release with the patch series #4 adds no red of its own. Everything else passes, including "Workflow lint and upstream drift".

finding: minor .github/workflows/fork-follow.yml:98 (also :289 and :305): adding "|| true" to the guard's run line (or to the re-check at :289, or to sweep_before's fork-workflows.sh at :305) leaves fork-follow.test.sh green, 104/104 (I tried all 3). With it on :98, the guard job succeeds after a refusal, the "passed" step sets the attempt, and plan and advance get FORK_PUSH_TOKEN. The new continue-on-error test guards only one of the two ways to swallow a refusal. This is a gap for a future edit, not a bug at this head. Suggested fix: assert that each refusal step's run is exactly the script line (no ||, no "set +e") (high)
finding: minor .github/scripts/fork-follow.test.sh:658 + .github/scripts/fork-gh-guard.sh:19: the guard tests cover a hand-written list (gh_scripts here, and fork-release.sh in its own file), and nothing checks that every .github/scripts/.sh calling gh sources fork-gh-guard.sh. Also, once GITHUB_ACTIONS=true and FORK_TEST_MODE is unset, the guard allows any gh. So: (1) a new script that forgets the source line, run by hand on a machine with a gh login, reaches GitHub. Its tests still hit only the stub, because fork_test_gh puts it first. (2) A new test file in CI that forgets fork_test_gh calls the runner's real gh: unauthenticated, because test steps get no GH_TOKEN, so reads only. Both need someone to forget something later; neither happens at this head. Suggested fix: one grep test that every .github/scripts/.sh calling gh outside comments sources fork-gh-guard.sh, and builds gh_scripts from that (medium)

@scuttletoby
scuttletoby merged commit 8b0ad2c into main Nov 12, 2026
20 of 22 checks passed
This was referenced Nov 12, 2026
Merged
Merged
Draft
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

size:L vouch:trusted PR author is trusted by repo permissions or the VOUCHED list.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@scuttletoby @noreply